Method and apparatus with data processing

Abstract

A processor-implemented data processing method includes normalizing input data of an activation function including a division operation, determining divisor data corresponding to a dividend of the division operation by reading a value of a first lookup table addressed according to the normalized input data from a memory, determining divisor data corresponding to a divisor of the division operation by accumulating the divisor data, and determining output data of the activation function corresponding to an output of the division operation, wherein the output of the division operation is obtained by reading a value of a second lookup table addressed according to the divisor data and the divisor data from the memory.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 915,846, filed October 16, 2019 with the United States Patent and Trademark Office, and to Korean Patent Application No. 10-2020-0054050, filed May 6, 2020 with the Korean Intellectual Property Office, the entire disclosures of which are incorporated herein by reference for all purposes. Technical Field

[0003] The following description relates to methods and apparatus for data processing. Background Technology

[0004] A neural network can be a computing system implemented using a reference computing architecture. Various electronic systems are used to analyze data and extract useful information by using data processing devices.

[0005] Data processing devices can perform a large number of data calculations. Summary of the Invention

[0006] This summary is provided to describe in simplified form the selection of concepts that will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0007] In one general aspect, a processor-implemented data processing method includes: normalizing input data of an activation function, including a division operation; determining dividend data corresponding to the dividend of the division operation by reading the value of a first lookup table addressed according to the normalized input data from memory; determining divisor data corresponding to the divisor of the division operation by accumulating the dividend data; and determining output data of the activation function corresponding to the output of the division operation, wherein the output of the division operation is obtained by reading the value of a second lookup table addressed according to the dividend data and the divisor data from memory.

[0008] The normalization may include: normalizing the input data so that the maximum value of the normalized input data is 0.

[0009] Determining the output data may include: determining a column index for addressing the columns of the second lookup table based on the value of the dividend data; determining a row index for addressing the rows of the second lookup table based on the value of the divisor data; and reading the values ​​of the second lookup table addressed according to the column index and the row index from the memory.

[0010] Determining the divisor data may include accumulating the dividend data using an accumulator, and determining the row index may include determining the row index from the value indicated by the most significant bit of the accumulator.

[0011] Determining the row index may include: determining the number of the most significant bits based on the base-2 logarithm of the total number of rows in the second lookup table.

[0012] Determining the number of the most significant bits may include adding 1 to the specific number of the logarithm value.

[0013] Determining the output data may include: reading the value of the second lookup table by selecting a word line of the memory addressed according to the row index.

[0014] The method may include: determining the precision of processing the data; and selecting a first lookup table and a second lookup table corresponding to the precision from a plurality of pre-generated lookup tables.

[0015] Selecting the first lookup table and the second lookup table may include selecting the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.

[0016] The activation function may include the Softmax function.

[0017] A non-transitory computer-readable storage medium that can store instructions, which, when executed by a processor, configure the processor to perform the method.

[0018] In another general aspect, a data processing apparatus includes: a memory; and a processor configured to: normalize input data of an activation function including a division operation; determine dividend data corresponding to the dividend of the division operation by reading the value of a first lookup table addressed according to the normalized input data from the memory; determine divisor data corresponding to the divisor of the division operation by summing the dividend data; and determine output data of the activation function corresponding to the output of the division operation, wherein the output of the division operation is obtained by reading the value of a second lookup table addressed according to the dividend data and the divisor data from the memory.

[0019] For the normalization, the processor can be configured to normalize the input data such that the maximum value of the normalized input data is 0.

[0020] To determine the output data, the processor can be configured to: determine a column index for addressing a column of the second lookup table based on the value of the dividend data; determine a row index for addressing a row of the second lookup table based on the value of the divisor data; and read the value of the second lookup table addressed according to the column index and the row index from the memory.

[0021] The apparatus may include an accumulator configured to accumulate the dividend data, wherein, for determining the row index, the processor may be configured to determine the row index from the value indicated by the most significant bit of the accumulator.

[0022] To determine the row index, the processor can be configured to determine the number of the most significant bits based on the base-2 logarithm of the total number of rows in the second lookup table.

[0023] The memory may include a dynamic random access memory (DRAM) storing the second lookup table, and for determining the output data, the processor may be configured to read the value of the second lookup table by selecting a word line of the DRAM addressed according to the row index.

[0024] The processor can be configured to: determine the precision of processing the data; and select a first lookup table and a second lookup table corresponding to the precision from a plurality of pre-generated lookup tables.

[0025] When selecting the first lookup table and the second lookup table, the processor can be configured to select the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.

[0026] The activation function may include the Softmax function.

[0027] In another general aspect, a processor-implemented data processing method includes: determining dividend data corresponding to the dividend of the activation function by reading the value of a first lookup table from memory based on input data of an activation function; determining divisor data corresponding to the divisor of the activation function by accumulating the dividend data; and determining output data corresponding to the output of the activation function by reading the value of a second lookup table from memory based on the dividend data and the divisor data.

[0028] The method may include: normalizing the input data, wherein determining the dividend data may include: reading the value of the first lookup table addressed according to the normalized input data from the memory.

[0029] The processor can be configured to perform automatic speech recognition based on the determined output data, and the activation function can correspond to the attention function.

[0030] In another general aspect, a processor-implemented data processing method includes: normalizing input data of an activation function, including a dividend and a divisor; determining dividend data by reading the value of a column index corresponding to the value of the normalized input data from a first lookup table of one or more memories; determining divisor data by summing the dividend data; and determining output data of the activation function by reading the value of a column index corresponding to the value of the normalized input data and the value of a row index corresponding to the value of the divisor data from a second lookup table of the one or more memories.

[0031] The dividend data can be in the range of 0 to 1.

[0032] The first lookup table and the second lookup table can be determined from multiple generated lookup tables based on the precision of the activation function that can be configured by the processor.

[0033] The precision can be any one of floating-point 10, floating-point 32, integer 8, integer 16, and unsigned integer 8.

[0034] Other features and aspects will become apparent from the following detailed description, drawings, and claims. Attached Figure Description

[0035] Figure 1 The architecture of a neural network according to one or more embodiments is shown.

[0036] Figure 2 A portion of an end-to-end automatic speech recognition (ASR) model is shown according to one or more embodiments.

[0037] Figure 3 A data processing apparatus according to one or more embodiments is shown.

[0038] Figure 4 A method for processing data according to one or more embodiments is shown.

[0039] Figure 5A and Figure 5B A method for implementing the Softmax function is shown using a data processing apparatus according to one or more embodiments.

[0040] Figure 6 The accuracy of the lookup table according to one or more embodiments is shown.

[0041] Figure 7 A lookup table is shown according to one or more embodiments.

[0042] Figure 8 A method for determining a row index is shown according to one or more embodiments.

[0043] Figure 9A and Figure 9B The architecture of a data processing apparatus according to one or more embodiments is shown.

[0044] Figure 10 The computational accuracy of a data processing apparatus according to one or more embodiments is shown.

[0045] Figure 11 The computational accuracy of the data processing apparatus according to an embodiment is shown.

[0046] Figure 12 A method is shown, according to one or more embodiments, for a data processing apparatus to read the value of a lookup table using dynamic random access memory (DRAM).

[0047] Figure 13 A method is shown for a data processing apparatus to compute a portion of a layer normalization function according to one or more embodiments.

[0048] Figure 14 A method for calculating a nonlinear function by a data processing apparatus according to one or more embodiments is shown.

[0049] Figure 15 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0050] Figure 16 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0051] Figure 17 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0052] Figure 18A and Figure 18B A method for performing a multiply-accumulate (MAC) operation by a data processing apparatus according to one or more embodiments is shown.

[0053] Figure 19 A method for processing arbitrary functions by a data processing apparatus according to one or more embodiments is shown.

[0054] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be drawn to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated. Detailed Implementation

[0055] The following detailed embodiments are provided to help readers gain a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order presented herein, but can be changed, as will become apparent after understanding the disclosure of this application, except for operations that must occur in a specific order. Moreover, after understanding the disclosure of this application, descriptions of features known in the art can be omitted to improve clarity and conciseness.

[0056] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein similar elements are indicated by similar reference numerals throughout the drawings. In this respect, one or more embodiments may have different forms and should not be construed as limited to the description set forth herein. Therefore, embodiments are described below only with reference to the accompanying drawings to illustrate multiple aspects. As used herein, the term “and / or” includes any and all combinations of one or more of the relevant listed items. When a statement such as “at least one of…” follows a list of elements, it modifies the entire list of elements, not individual elements within the list.

[0057] The terminology used in this disclosure has been chosen with regard to the functionality of this disclosure and based on currently widely used general terminology. However, once understood, the terminology may vary according to the intent of those skilled in the art, precedent, or new technologies in the field. Furthermore, the applicant may arbitrarily choose some terms, and in such cases, the meaning of the chosen terms will be described in the specific embodiments of this disclosure. Therefore, the terminology used herein should not be interpreted solely based on the name of the term, but rather on its meaning and the description throughout this disclosure.

[0058] The terminology used herein is for describing various examples only and is not intended to limit this disclosure. Unless the context clearly indicates otherwise, the articles “a,” “an,” and “the” are also intended to include plural forms. The terms “comprising,” “including,” and “having” specify the presence of said features, numbers, operations, members, elements, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, operations, members, elements, and / or combinations thereof. Terms such as “unit” used in embodiments indicate a unit for processing at least one function or operation, and wherein the unit is hardware or a combination of hardware and software.

[0059] This disclosure will now be described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0060] Figure 1 The architecture of a neural network 1 according to one or more embodiments is shown.

[0061] refer to Figure 1 Neural network 1 can be implemented as a deep neural network (DNN) or an n-layer neural network. A DNN or an n-layer neural network can correspond to a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network, a restricted Boltzmann machine, etc. For example, neural network 1 can be implemented as a CNN, but is not limited to this. Figure 1 Some convolutional layers in a CNN corresponding to the example of Neural Network 1 are shown, but CNNs can also include pooling layers and fully connected layers in addition to the convolutional layers shown.

[0062] Neural Network 1 can be implemented as a multi-layered computational architecture, comprising an input image layer, a feature map generation layer, and an output data layer. In Neural Network 1, when a convolution operation is performed between the input image and a filter called a weight kernel, an output feature map (or activation map or convolved feature) can be output. Activation functions can be used during the generation of the output feature map. The generated output feature map can be the next input feature map for the next convolutional layer, and a convolution operation can be performed between this next input feature map and the kernel of the next convolutional layer, resulting in a new output feature map. As a result of repeatedly performing such convolution operations using various kernels, the final output can be a result of recognizing features of the input image through Neural Network 1.

[0063] For example, when to Figure 1 When the neural network 1 receives an image of size 24×24 pixels as input, it outputs a four-channel feature map of size 20×20 pixels through a first convolution operation between the input image and a first kernel. Subsequently, the size of the corresponding output feature map is gradually reduced by performing various convolution operations on the output 20×20 feature map with each kernel, generating a final feature map of size 1×1 pixels using the final convolution operation with the last kernel. The neural network 1 can filter and output robust features representing the entire image by performing convolution operations and downsampling (or pooling) operations accordingly in each layer, and can obtain a recognition result about the input image from the final output features. Based on the above, it should be understood that the exemplary embodiments discussed herein include embodiments having one or more, or all, such layers, levels, edges, etc., in the operation of the neural network 1, or various combinations thereof.

[0064] Figure 2 A portion of an end-to-end automatic speech recognition (ASR) model is shown according to one or more embodiments.

[0065] ASR (Automatic Speech Representation) is a technique that converts human speech into data that computers can interpret using artificial intelligence (AI). ASR models can be implemented based on neural networks. For example, ASR models can be implemented based on DNNs, recurrent neural networks (RNNs), etc. Additionally, ASR models can be implemented based on various algorithms such as Hidden Markov Models (HMMs).

[0066] An end-to-end ASR model can be an ASR model that directly maps an input sequence obtained from speech into a sequence of words.

[0067] End-to-end ASR models can be implemented based on attention mechanisms. Attention mechanisms map input sequences to output sequences using the dependencies between them. Attention mechanisms can be implemented using methods including, for example, scaled dot product attention functions.

[0068] In an end-to-end ASR model, the scaling dot product attention function can be processed by a computation module or processing element (e.g., one or more processors). The scaling dot product attention function can be a function used to map a set of query Q, key K, and value V to an output, and can be expressed, for example, as Equation 1 below, and in Figure 2 As shown in the image.

[0069] Equation 1:

[0070]

[0071] In Equation 1, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and dk represents the dimension of the query and key (e.g., the query and key with dimension dk).

[0072] The association between the query and the key can be obtained (e.g., determined) from the dot product of the query and the key. This is achieved by... Scaling the dot product of the query and the key prevents slight changes to the output of the Softmax function. The higher the correlation between the output of the Softmax function and the query, the greater the Attention(Q, K, V) can be obtained.

[0073] Figure 3 This is a configuration diagram showing a data processing apparatus 300 according to one or more embodiments.

[0074] refer to Figure 3The data processing device 300 may include a memory 310 and a processor 320 (e.g., one or more processors). Figure 3 In the data processing apparatus 300, components related to this embodiment are shown, and it will be clear upon understanding this disclosure that, in addition to Figure 3 In addition to the components shown, the data processing device 300 may also include other components.

[0075] The data processing device 300 can be used to implement a reference. Figure 1 The neural network apparatus described above. Additionally, the data processing unit 300 can be a reference implementation. Figure 2 The apparatus described above is part of the ASR model. For example, the data processing apparatus 300 can be implemented as or utilized by various types of devices, such as personal computers (PCs), server devices, mobile devices, embedded devices, etc. As specific examples, the data processing apparatus 300 can be or includes smartphones, tablet devices, augmented reality (AR) devices, Internet of Things (IoT) devices, autonomous vehicles, robotic devices, medical devices, etc., which use neural networks to perform speech recognition, image recognition, and image classification, but are not limited thereto. Furthermore, the data processing apparatus 300 can correspond to a dedicated hardware accelerator installed on the aforementioned devices, and can be a hardware accelerator such as a neural processing unit (NPU), a tensor processing unit (TPU), a neural engine, etc., which are dedicated modules for driving neural networks.

[0076] The memory 310 can store various types of data processed in the data processing device 300. For example, the memory 310 can store data that has been processed by the data processing device 300 and data that will be processed by the data processing device 300. For example, the memory 310 can be used as an accumulator by storing data processed by the data processing device 300. In addition, the memory 310 can store applications, drivers, etc., that are to be driven by the data processing device 300. For example, the memory 310 can be an on-chip memory responsible for caching functions to process operations.

[0077] For example, memory 310 may include random access memory (RAM) (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid-state drive (SSD) and / or flash memory.

[0078] The processor 320 can control the overall functions used to drive the neural network in the data processing device 300. For example, the processor 320 can execute a program stored in the memory 310 to control the data processing device 300 overall. The processor 320 can be implemented as a central processing unit (CPU), graphics processing unit (GPU), application processor (AP), etc., provided in the data processing device 300, but is not limited thereto.

[0079] Processor 320 can read / write data (e.g., voice data, image data, feature map data, kernel data, etc.) from / to memory 310 and use the read / written data to process neural network operations. Additionally, processor 320 can read / write voice data from / to memory 310 and use the read / written data to process ASR operations.

[0080] Processor 320 may include processing elements for processing operations. For example, the processing element may include logic circuitry for arithmetic operations. For example, the processing element may include an arithmetic unit implemented as a combination of multipliers and adders. Alternatively, the processing element may include an arithmetic unit implemented as a combination of multipliers, adders, and accumulators. Furthermore, a multiplier may be implemented as a combination of multiple sub-multipliers, and an adder may also be implemented as a combination of multiple sub-adders.

[0081] Processor 320 may also include an allocator for allocating operands. For example, the allocator may allocate operands from data stored in memory 310 to memory 310 for operations to be performed by the processing element. The allocator may then allocate back to the processing element for computation on the operands allocated to memory 310.

[0082] In the following text, reference will be made to Figures 4 to 19 This describes an embodiment relating to a method for processing data using a data processing apparatus 300. It can be applied to... Figure 3 The content described for the data processing apparatus 300 shown applies to the embodiments described below.

[0083] Figure 4 This is a flowchart illustrating a method for processing data according to one or more embodiments.

[0084] Figure 4 The method for processing data illustrates how the data processing apparatus 300 can implement an activation function. In one or more embodiments, the activation function may include a division operation.

[0085] In operation 410, processor 320 can normalize the input data of activation functions, including division operations.

[0086] The input data can be data used as input to an activation function. Processor 320 can normalize the input data by performing a preprocessing procedure to normalize the input data to a specific rule. For example, processor 320 can normalize the input data such that the maximum and / or minimum values ​​of the input data are included within a preset range. As another example, processor 320 can normalize the input data such that the maximum and / or minimum values ​​of the input data are preset values.

[0087] In operation 420, processor 320 can obtain (e.g., determine) the dividend data corresponding to the dividend of the division operation by reading the value of a first lookup table addressed according to normalized input data from memory 310.

[0088] The processor 320 can obtain the dividend data by reading the value of a first lookup table addressed according to the normalized input data. The dividend data can be data corresponding to the dividend in the division operation. The dividend data may not have the same value as the dividend in the division operation.

[0089] In operation 430, processor 320 can obtain divisor data corresponding to the divisor in the division operation by accumulating the dividend data.

[0090] The processor 320 can obtain the divisor data by accumulating the dividend data using an accumulator or adder included in the processing element. Alternatively, the processor 320 can obtain the divisor data by accumulating the dividend data using an accumulator included in the memory 310.

[0091] Processor 320 obtains divisor data by summing the dividend data instead of directly calculating the divisor of the division operation. The divisor data can be data corresponding to the divisor of the division operation. The divisor data may not have the same value as the divisor of the division operation.

[0092] In operation 440, processor 320 can obtain the output data of the activation function from the output of the division operation, wherein the output of the division operation is obtained by reading the value of a second lookup table addressed according to the dividend data and the divisor data from memory 310.

[0093] Instead of using a divider, processor 320 can obtain the output of the division operation by reading the values ​​of a second lookup table addressed to the dividend and divisor data. Processor 320 can then obtain output data from the division operation. The output data may be the output of an activation function.

[0094] In another embodiment, the normalization operation 410 on the input data can be omitted. Instead of normalizing the input data, the values ​​of the first and second lookup tables can be fine-tuned, thereby obtaining the output of the division operation of the activation function within a low error range.

[0095] Figure 5A and Figure 5B This is a diagram illustrating a method of implementing the Softmax function using a data processing apparatus according to one or more embodiments.

[0096] The Softmax function can be an activation function that can be used to implement neural networks and / or ASR. In the examples, the Softmax function can correspond to the Softmax function in Equation 1 described above. For example, the Softmax function can be expressed as Equation 2 below.

[0097] Equation 2:

[0098]

[0099] refer to Figure 5A and Figure 5B The processor 320 can normalize the input data x. In one or more embodiments, the processor 320 can normalize the input data x such that the normalized input data... The maximum value is 0. In one or more embodiments, for example, processor 320 may normalize the input data x based on the following equation 3.

[0100] Equation 3:

[0101]

[0102] In equation 3, Let x represent the normalized input data, and max(x) represent the maximum value of the input data.

[0103] When normalized input data When the maximum value is 0, for the normalized input data The result of performing the exponentiation operation is included in Figure 5B The shaded area in the curve graph. That is, the dividend data. It is included in the range of 0 to 1.

[0104] The result of exponential operations on input data x is included in the range from 0 to ∞, while the result of normalized input data... The result of performing an exponential operation is included in the range of 0 to 1. Therefore, the dividend of the Softmax function for the input data x is included in the range of 0 to ∞, and the dividend data... It is included in the range of 0 to 1. Therefore, due to the divisor data... Since the values ​​are distributed in the range of 0 to 1, the values ​​in the first lookup table can be stabilized within a finite range. For example, while a typical data processing apparatus can implement a lookup table with values ​​in the range of 0 to ∞, a data processing apparatus of one or more embodiments can implement normalized input data distributed in the range of 0 to 1. Dividend data The first lookup table for the values ​​is such that the size of the storage space (e.g., memory 310) used to store the first lookup table implemented by the data processing apparatus of one or more embodiments is advantageously reduced compared to the size of the storage space used to store the first lookup table implemented by a typical data processing apparatus.

[0105] Processor 320 can read normalized input data The value of the first lookup table (first LUT) is used for addressing to obtain (e.g., determine) the dividend data corresponding to the dividend of the Softmax function. For example, the dividend of the Softmax function can be related to the one in Equation 2 above. Correspondingly.

[0106] For example, when the input data x consists of n elements x1, x2, ..., x... n When processing a vector, the processor 320 can read the normalized input data from the memory 310 n times. The first lookup table value is used for addressing, resulting in n elements. Dividend data

[0107] Processor 320 can process the dividend data The summation is performed to obtain the divisor data corresponding to the divisor of the Softmax function. For example, the divisor of the Softmax function can be the same as the divisor in Equation 2 above. Correspondingly.

[0108] For example, when the input data x consists of n elements x1, x2, ..., x... n The vector was obtained and contained n elements. Dividend data At that time, the processor 320 can process the dividend data n elements The divisor data is obtained by summing the results.

[0109] Processor 320 can read data based on the dividend from memory 310. The output data σ of the Softmax function is obtained by using the value of the second lookup table (second LUT) addressed to the divisor data. In other words, the processor 320 can read the data from the memory 310 based on the dividend data... Instead of directly performing the division operation of the Softmax function, the value of the second lookup table (second LUT) addressing the divisor data is used to obtain the output of the division operation. For example, the output of the Softmax function can be compared with σ(x) in Equation 2 above. i This corresponds to, for example, the output data σ, which can be related to σ(x) in Equation 2 above. i Corresponding to.

[0110] For example, when the input data x consists of n elements x1, x2, ..., x... n The vector was obtained and contained n elements. Dividend data At that time, the processor 320 can read the dividend data from the memory 310 n times. The value of the second lookup table, which is used to address the divisor data, is used to obtain n elements σ(x1), σ(x2), ..., σ(x... n The output data σ.

[0111] When the input data x is normalized so that the maximum value is 0, the dividend data... It can be included in the range of 0 to 1, regardless of the range of the input data x. Therefore, by considering the dividend data... The range of values ​​for the divisor obtained by accumulation depends on the length of the input data x.

[0112] In a data processing method according to one or more embodiments, processor 320 can obtain the output of the activation function by performing n-cycle operations of reading the values ​​of a first lookup table and n-cycle operations of reading the values ​​of a second lookup table. The accumulation of the dividend data can be performed in the same cycle as reading the values ​​of the first lookup table, therefore processor 320 can calculate the activation function through 2n-cycle operations.

[0113] To implement neural networks and / or ASR, activation functions, including division, can be processed. Typical data processing devices use dedicated arithmetic units to implement division operations for activation functions. For example, when the Softmax function includes division, a typical data processing device can use a divider to implement the division operation. The dedicated arithmetic units used for activation functions in typical data processing devices may be non-reconfigurable, and the divider may cause latency in computation.

[0114] Conversely, the data processing apparatus 300 of one or more embodiments can use a lookup table (e.g., either or both of a first lookup table and a second lookup table) to process activation functions, including division operations. Therefore, the data processing apparatus 300 of one or more embodiments can avoid using a dedicated arithmetic unit for processing activation functions and can be configured to be reconfigurable. Furthermore, because the data processing apparatus 300 can perform division operations without using a divider, there is no latency caused by such a divider, thereby improving functionality in the fields of computer and data processing, neural network processing, and ASR.

[0115] Figure 6 The accuracy of the lookup table according to one or more embodiments is shown.

[0116] The precision with which the processor 320 (and / or neural processor or neural processing unit (NPU)) computes, operates, or implements the activation function can vary. For example, the processor 320 may compute the activation function with a precision of floating-point (FP) 32, integer (INT) 16, INT 8, or unsigned integer (UINT) 8.

[0117] Multiple lookup tables 600 with various precisions can be pre-generated and stored in memory 310. The multiple lookup tables 600 may include a 1D lookup table 1D_LUT_… used as a first lookup table and a 2D lookup table 2D_LUT_… used as a second lookup table.

[0118] For example, memory 310 may store multiple lookup tables 610 having a precision of FP10 corresponding to the precision of FP32 of the NPU, multiple lookup tables 620 having a precision of INT16 corresponding to the precision of INT16 of the NPU, multiple lookup tables 630 having a precision of INT8 corresponding to the precision of INT8 of the NPU, and multiple lookup tables 640 having a precision of UINT8 corresponding to the precision of UINT8 of the NPU. Processor 320 may include an NPU.

[0119] Various lookup tables with the same precision can be stored in memory 310. For example, lookup tables with the same precision but different lengths (columns / rows) and sizes (bits / bytes) can be stored in memory 310. For example, a 31-column one-dimensional lookup table 1D_LUT_1x31 with a precision of FP10, a 61-column one-dimensional lookup table 1D_LUT_1x61, etc., can be stored in memory 310. For example, a 275-byte two-dimensional lookup table 2D_LUT_11x20 with a precision of FP10, a 550-byte two-dimensional lookup table 2D_LUT_11x40, etc., can be stored in memory 310.

[0120] Processor 320 can select a first lookup table and a second lookup table from a plurality of lookup tables 600 based on the precision of the operation to be performed. Furthermore, processor 320 can select first and second lookup tables with the same or different precisions. For example, processor 320 can select a 1D lookup table 1D_LUT_1x101_int8 with a precision of INT8 as the first lookup table, and a 2D lookup table 2D_LUT_11x60_int8 with a precision of INT8 as the second lookup table. For example, processor 320 can select a 1D lookup table 1D_LUT_1x61 with a precision of FP10 as the first lookup table, and a 2D lookup table 2D_LUT_11x60_int8 with a precision of INT8 as the second lookup table.

[0121] Multiple lookup tables 600 can be stored in DRAM, and the first and second lookup tables selected according to precision can be moved and stored in SRAM. When the first and second lookup tables to be used to calculate the activation function are stored in SRAM, the processor 320 can read the values ​​of the first and second lookup tables from the memory 310 at high speed.

[0122] Figure 7 A lookup table is shown according to one or more embodiments.

[0123] In one or more embodiments, it can be as follows Figure 7 The diagram shows the generation of a 1D lookup table 1D_LUT_1x101_uint8 with a precision of UINT8. The 1D lookup table 1D_LUT_1x101_uint8 can have 101 columns.

[0124] The values ​​in the 1D lookup table 1D_LUT_1x101_uint8 can be values ​​corresponding to the result of performing an exponential operation on normalized input data. When the input data is normalized such that the maximum value is 0, the result of performing an exponential operation on the normalized input data is included in the range of 0 to 1. Therefore, a 1D lookup table 1D_LUT_1x101_uint8 can be generated to represent the range of 0 to 1 with a precision of UINT8.

[0125] The processor 320 can obtain (e.g., determine) the dividend data by obtaining the column index for addressing the columns of the 1D lookup table 1D_LUT_1x101_uint8 based on the value of the normalized input data, and by referring to the column index to read the value of the 1D lookup table 1D_LUT_1x101_uint8.

[0126] In one or more embodiments, it can be as follows Figure 7The diagram shows the generation of a 2D lookup table 2D_LUT_11x60_uint8 with a precision of UINT8. The 2D lookup table 2D_LUT_11x60_uint8 can have 11 rows and 60 columns.

[0127] Processor 320 can obtain column indices for addressing columns of the 2D lookup table 2D_LUT_11x60_uint8 based on the values ​​of the normalized input data, and row indices for addressing rows of the 2D lookup table 2D_LUT_11x60_uint8 based on the values ​​of the divisor data. Processor 320 can obtain the output data of the activation function by reading the values ​​of the 2D lookup table 2D_LUT_11x60_uint8 by referring to the column and row indices.

[0128] Figure 8 A method for determining a row index is shown according to one or more embodiments.

[0129] Processor 320 can obtain the row index for addressing rows in the second lookup table based on the value of the divisor data. For example, processor 320 can obtain the divisor data by accumulating the dividend data using accumulator 800, and processor 320 can obtain the row index from certain bits of accumulator 800 (e.g., certain bits of the accumulated dividend data).

[0130] Processor 320 can obtain row indexes from the most significant bit 810 of a specific or determined number from accumulator 800. Processor 320 can determine the specific number based on the number of rows in a second lookup table.

[0131] The processor 320 can determine the specific quantity based on the base-2 logarithm of the row number of the second lookup table. For example, the processor 320 can determine the specific quantity based on the index value of the base-2 logarithm of the row number of the second lookup table. The processor 320 can also determine the specific quantity by adding 1 to the base-2 logarithm of the row number of the second lookup table.

[0132] For example, when the number of rows in the second lookup table is 30, because the value of log230 is approximately 4.91, the processor 320 can determine the specific number as the value 5 obtained by incrementing index 4 by 1, and can obtain the row index from the 5 most significant bits of the accumulator 800. For example, when the number of rows in the second lookup table is 60, because the value of log260 is approximately 5.91, the processor 320 can determine the specific number as the value 6 obtained by incrementing index 5 by 1, and can obtain the row index from the 6 most significant bits of the accumulator 800.

[0133] Figure 9A and Figure 9BThis is a diagram illustrating the architecture of a data processing apparatus (e.g., data processing apparatus 300) according to one or more embodiments.

[0134] refer to Figure 9A and Figure 9B The processing elements (PEs) 911 and 912 of the data processing apparatus 300 may include an arithmetic unit implemented as a combination of multipliers and adders. Alternatively or additionally, the processing elements 911 and 912 may include an arithmetic unit implemented as a combination of multipliers, adders, and accumulators. The processor 320 may include the processing elements 911 and 912.

[0135] Dedicated, reconfigurable memories 921 and 922 can be allocated to processing elements 911 and 912. For example, dedicated memory 921 can be allocated one-to-one to processing element 911, or dedicated memory 922 can be allocated many-to-one to processing element 912. Memory 310 may include dedicated memories 921 and 922.

[0136] When processing elements 911 and 912 include accumulators, processing elements 911 and 912 can obtain divisor data by accumulating dividend data. Alternatively, dedicated memories 921 and 922 can obtain divisor data by accumulating dividend data.

[0137] The first lookup table and the second lookup table can be stored in dedicated memories 921 and 922. The processor 320 can read the values ​​of the first lookup table and the second lookup table from the dedicated memories 921 and 922.

[0138] Dedicated memories 921 and 922 may have a size sufficient to store the first lookup table and the second lookup table. For example, dedicated memory 921, allocated one-to-one to processing element 911, may be 256 bytes. For example, dedicated memory 922, allocated many-to-one to processing element 912, may be 32K bytes.

[0139] The dedicated memories 921 and 922 allocated to the processing elements 911 and 912 may be SRAM, enabling the processor 320 to read the first and second lookup tables at high speed.

[0140] Processor 320 can use processing elements 911 and 912 to perform operations between activations and weights of a neural network, and can use dedicated memories 921 and 922 to perform division operations on activation functions. For example, processor 320 can use processing elements 911 and 912 to perform convolution operations between activations and weights, and can perform division operations on activation functions by reading a first lookup table and a second lookup table from dedicated memories 921 and 922.

[0141] Additionally, processor 320 can use processing elements 911 and 912 to perform dot product operations for scaling dot product attention functions to implement end-to-end ASR, and can use dedicated memories 921 and 922 to perform operations of the Softmax function.

[0142] Figure 10 The computational accuracy of a data processing apparatus according to one or more embodiments is shown.

[0143] Experiments were conducted to measure the accuracy of speech recognition for an end-to-end ASR model based on a scaled dot product attention function. The experiments were performed using 1000 short samples from the LibriSpeech dataset and approximately 2000 samples from a commercial dataset. Short samples are those with a word rate per sentence (WPSR) of 10 or less.

[0144] In the experiments, the accuracy of speech recognition was compared between a typical reference model 1010, which calculates the Softmax function using a conventional method with an accuracy of FP64 (using a dedicated arithmetic unit for division operations to implement the Softmax function and / or without using the first and second lookup tables according to one or more embodiments as described above), and a model of one or more embodiments of the proposed method for calculating the Softmax function according to the method shown in Figure 5. The accuracy of speech recognition was calculated based on word error rate (WER) and sentence error rate (SER).

[0145] The proposed models of one or more embodiments are configured to compute the Softmax function using a first lookup table and a second lookup table of different lengths (rows / columns) with an accuracy of FP10. For example, the proposed first model 1020 is configured to use a first lookup table with 31 columns and a second lookup table with 40 rows and 11 columns, the proposed second model 1030 is configured to use a first lookup table with 61 columns and a second lookup table with 60 rows and 11 columns, and the proposed third model 1040 is configured to use a first lookup table with 101 columns and a second lookup table with 60 rows and 11 columns.

[0146] As an experimental result, the WER of the proposed model in one or more embodiments increased by only about 0.05% to 0.5% compared to the typical reference model 1010. Furthermore, the SER of the proposed model increased by only about 0.1% to 2% compared to the typical reference model 1010. For example, in the case of the proposed first model 1020 using first and second lookup tables of size 600 bytes or less, the WER increased by only about 0.2% to 0.5% compared to the typical reference model 1010. Therefore, when using the proposed model in one or more embodiments, accurate speech recognition can be performed smoothly with fewer hardware resources than the typical reference model 1010, thereby improving the functionality of computer and data processing, neural network processing, and ASR technologies.

[0147] Figure 11 The computational accuracy of a data processing apparatus according to one or more embodiments is shown.

[0148] Experiments were conducted to measure the speech recognition accuracy of models using one or more embodiments of the proposed first and second lookup tables with the same precision and models using one or more embodiments of the proposed first and second lookup tables with different precision. Figure 11 Based on reference Figure 10 The experimental results for measuring the accuracy of the proposed model are shown in the typical reference model 1010 described above.

[0149] The proposed model of one or more embodiments includes: case full_int16, which selects a 1D lookup table 1D_LUT_1x101_int16 with precision INT16 as the first lookup table and a 2D lookup table 2D_LUT_11x60_int16 with precision INT16 as the second lookup table; case full_uint8, which selects a 1D lookup table 1D_LUT_1x101_uint8 with precision UINT8 as the first lookup table and a 2D lookup table 2D_LUT_11x60_uint8 with precision UINT8 as the second lookup table; and case full_int8, which selects a 1D lookup table 1D_LUT_1x101_int8 with precision INT8 as the first lookup table and a 2D lookup table 2D_LUT_11x60_int8 with precision INT8 as the second lookup table.

[0150] Additionally, the model of one or more of the proposed embodiments includes: case mix_16_8, which selects a 1D lookup table 1D_LUT_1x101_int16 with a precision of INT16 as the first lookup table and a 2D lookup table 2D_LUT_11x60_int8 with a precision of INT8 as the second lookup table.

[0151] As the results of the experiments, it can be confirmed that the WER of the proposed model (one or more embodiments) differs from that of the typical reference model 1010 by only 0 to 1%. Therefore, when using the proposed model (one or more embodiments), speech recognition can be performed with similar accuracy to the typical reference model 1010, while using fewer hardware resources than the typical reference model 1010, thereby improving the functionality of the fields of computer and data processing, neural network processing, and ASR.

[0152] Figure 12 This invention illustrates a method by which a data processing apparatus, according to one or more embodiments, reads the value of a lookup table using DRAM.

[0153] Processor 320 may read a 2D lookup table from memory 310. In one or more embodiments, processor 320 may use a 2D lookup table stored in SRAM, but in another embodiment, processor 320 may use a 2D lookup table stored in DRAM.

[0154] The processor 320 can read the value of the 2D lookup table by selecting the word lines of the DRAM that are addressed by the row index of the 2D lookup table and the bit lines of the DRAM that are addressed by the column index of the 2D lookup table.

[0155] Because only one word line selection is needed in the DRAM to read the value from the 2D lookup table, the value of the 2D lookup table can be obtained at a relatively high speed even when using DRAM instead of SRAM.

[0156] Figure 13 A method is shown for calculating a portion of a layer normalization function using a data processing apparatus according to one or more embodiments.

[0157] For example, a portion of the layer normalization function can be expressed as Equation 4 below.

[0158] Equation 4:

[0159]

[0160] In Equation 4, y represents the part of the layer normalization function corresponding to the output data, x represents the input data, and n represents the number of elements in x.

[0161] Processor 320 can obtain the square x1 of the elements of input data x by reading the value of the first lookup table (first LUT) addressed according to input data x from memory 310. 2 x2 2 ... x n 2 Processor 320 can process the elements of input data x by squaring x1.2 x2 2 ... x n 2 By accumulating the elements, we obtain the square of the elements x1 of the input data x. 2 x2 2 ... x n 2 The sum. The squares of the elements x1 of the input data x. 2 x2 2 ... x n 2 The sum can be combined with ∑x in equation 4 i 2 Correspondingly, processor 320 can read the square of the elements x1 according to the input data x from memory 310. 2 x2 2 ... x n 2 The values ​​of the second lookup table (second LUT) are used to address the sum of the elements and the number of elements n of the input data x, in order to obtain the output data corresponding to that part of the layer normalization function.

[0162] Figure 14 A method for calculating a nonlinear function is shown using a data processing apparatus according to one or more embodiments.

[0163] Processor 320 can calculate a nonlinear function with respect to the input data by reading the values ​​of a lookup table addressed by the input data from memory 310. In other words, processor 320 can calculate the nonlinear function with respect to the input data by reading the values ​​of the lookup table, rather than directly calculating the nonlinear function with respect to the input data.

[0164] In one or more embodiments, the processor 320 can read input data x1, x2, ..., x... n The values ​​of the lookup table (LUT) are used for addressing to obtain information about the input data x1, x2, ..., x. n The value of the hyperbolic tangent. In addition, the processor 320 can obtain data y1, y2, ..., yn, which are obtained by inverse quantizing the value of the hyperbolic tangent by scaling the obtained value of the hyperbolic tangent.

[0165] Figure 15 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0166] The processor 320 can perform multiplication operations by reading the values ​​of a 2D lookup table from the memory 310.

[0167] In one or more embodiments, the 2D lookup table may include the product of numbers that can be represented using 4 bits. The 2D lookup table may be a lookup table with 16 rows and 16 columns. Therefore, the processor 320 of one or more embodiments can obtain the product of numbers by reading the values ​​of the 2D lookup table, rather than (e.g., using a dedicated multiplier) directly calculating the multiplication between numbers.

[0168] Figure 16 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0169] In multiplication, even if the order of the multipliers changes, the product remains the same, so the processor 320 can read the product... Figure 16 The values ​​shown in the 2D lookup table are not... Figure 15 The 2D lookup table shown is used to perform multiplication. This is achieved by using... Figure 16 The 2D lookup table can save storage space in memory 310.

[0170] Figure 17 A method for performing multiplication operations by a data processing apparatus according to one or more embodiments is shown.

[0171] Processor 320 can perform multiplication operations between numbers represented in different bits by reading the values ​​of a 2D lookup table from memory 310. In one or more embodiments, processor 320 can obtain the multiplication result between a number that can be represented in 4 bits and a number that can be represented in 8 bits by reading the values ​​of the 2D lookup table.

[0172] Figure 18A and Figure 18B A method for performing a multiply-accumulate (MAC) operation is shown in a data processing apparatus according to one or more embodiments.

[0173] Processor 320 can perform MAC operations. In one or more embodiments, processor 320 can perform multiplication by reading values ​​from a 2D lookup table 1810 from memory 310 and perform addition by using accumulator 1820 to perform MAC operations. In another embodiment, processor 320 can perform MAC operations by reading values ​​from a 3D lookup table 1830.

[0174] Figure 19 A method for processing arbitrary functions is shown using a data processing apparatus according to one or more embodiments.

[0175] Processor 320 can process arbitrary functions by reading a lookup table from memory 310. The arbitrary function can be a linear function or a nonlinear function. The arbitrary function can include linear operations or nonlinear operations.

[0176] In one or more embodiments, processor 320 can obtain the output data f(a, b) of any function f with respect to input data a and b by reading the values ​​of a 2D lookup table 1910 from memory 310. Instead of directly calculating the function, processor 320 can obtain the output data f(a, b) of the function by using lookup tables of various dimensions such as 1D, 2D, and 3D.

[0177] Data processing device, memory, processor, accumulator, processing element, NPU, data processing device 300, memory 310, processor 320, accumulator 800, processing element 911, processing element 912, memory 921, memory 922, accumulator 1820, and for... Figures 1 to 19Other devices, apparatuses, units, modules, and components described herein are implemented by or represent hardware components. Examples of hardware components that may be used, where appropriate, to perform the operations described herein include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described herein. In other examples, one or more hardware components performing the operations described herein are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented by one or more processing elements (e.g., logic gate arrays, controllers, and arithmetic logic units), digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, a processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer may execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, to perform the operations described herein. Hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, while one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. Hardware components may have any one or more of different processing configurations, examples of which include single-processor, standalone processor, parallel processor, single-instruction single-data (SISD) multiple processing, single-instruction multiple-data (SIMD) multiple processing, multiple-instruction single-data (MISD) multiple processing, and multiple-instruction multiple-data (MIMD) multiple processing.

[0178] exist Figures 1 to 19The methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., one or more processors or computers equipped with execution instructions or software implemented as described above to perform the operations described in this application performed by the methods). For example, a single operation or two or more operations may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, may execute a single operation, or two or more operations.

[0179] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof, for individually or collectively instructing or configuring one or more processors or computers as machines or special-purpose computers to perform the operations performed by the hardware components and methods as described above. In one example, the instructions or software include machine code that is directly executed by one or more processors or computers, for example, machine code generated by a compiler. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions used herein, which disclose algorithms for performing the operations performed by the hardware components and methods as described above.

[0180] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed on or in one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), flash memory, card memory (e.g., multimedia microcard) or card (e.g., Secure Digital (SD) or Ultimate Digital (XD)), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state disk, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide said instructions or software and any associated data, data files, and data structures to one or more processors or computers so that said one or more processors or computers can execute said instructions. In one example, said instructions or software and any associated data, data files, and data structures are distributed across a network-connected computer system, such that said instructions or software and any associated data, data files, and data structures are stored, accessed, and executed by one or more processors or computers in a distributed manner.

[0181] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example should be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if the components in the described system, architecture, device, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

Claims

1. A processor-implemented data processing method for processing division operations included in the activation function of a neural network without any delay caused by the divider, the method comprising: The input data of the activation function, including the division operation, is normalized; By reading the value of a first lookup table addressed according to normalized input data from memory, the dividend data corresponding to the dividend in the division operation is determined; By accumulating the dividend data using an accumulator, the divisor data corresponding to the divisor in the division operation is determined; and The output data of the activation function corresponding to the output of the division operation is determined, wherein the output of the division operation is obtained by reading the value of a second lookup table addressed according to the dividend data and the divisor data from the memory. The determination of the output data includes: Based on the value of the dividend data, determine the column index for addressing the column of the second lookup table; The row index for addressing a row in the second lookup table is determined from the value of the divisor data indicated by the most significant bit of the accumulator; and The value of the second lookup table, addressed according to the column index and the row index, is read from the memory.

2. The method according to claim 1, wherein, The normalization includes: normalizing the input data so that the maximum value of the normalized input data is 0.

3. The method according to claim 1, wherein, Determining the row index includes: determining the number of the most significant bits based on the base-2 logarithm of the total number of rows in the second lookup table.

4. The method according to claim 3, wherein, Determining the number of the most significant bits includes adding 1 to the specific number of the logarithm value.

5. The method according to claim 1, wherein, Determining the output data includes: reading the value of the second lookup table by selecting the word line of the memory addressed according to the row index.

6. The method according to claim 1, further comprising: Determine the precision required for data processing; as well as Select the first lookup table and the second lookup table corresponding to the precision from a plurality of pre-generated lookup tables.

7. The method according to claim 6, wherein, Selecting the first lookup table and the second lookup table includes: Select the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.

8. The method according to claim 1, wherein, The activation function includes the Softmax function.

9. A non-transitory computer-readable storage medium for storing instructions, wherein when the instructions are executed by a processor, the processor is configured to perform the method of claim 1.

10. A data processing apparatus for processing division operations included in the activation function of a neural network without any delay caused by the divider, the apparatus comprising: Memory; as well as The processor is configured as follows: The input data of the activation function, including the division operation, is normalized; By reading the value of a first lookup table addressed according to normalized input data from the memory, the dividend data corresponding to the dividend in the division operation is determined; By accumulating the dividend data using an accumulator, the divisor data corresponding to the divisor in the division operation is determined; and The output data of the activation function corresponding to the output of the division operation is determined, wherein the output of the division operation is obtained by reading the value of a second lookup table addressed according to the dividend data and the divisor data from the memory. Specifically, to determine the output data, the processor is configured as follows: Based on the value of the dividend data, determine the column index for addressing the column of the second lookup table; The row index for addressing the row of the second lookup table is determined from the value of the divisor data indicated by the most significant bit of the accumulator; as well as The value of the second lookup table, addressed according to the column index and the row index, is read from the memory.

11. The apparatus according to claim 10, wherein, For the normalization, the processor is configured to normalize the input data such that the maximum value of the normalized input data is 0.

12. The apparatus according to claim 10, wherein, To determine the row index, the processor is configured to: determine the number of the most significant bits based on the base-2 logarithm of the total number of rows in the second lookup table.

13. The apparatus according to claim 10, wherein: The memory includes: a dynamic random access memory (DRAM) storing the second lookup table, and To determine the output data, the processor is configured to read the value of the second lookup table by selecting a word line of the DRAM addressed according to the row index.

14. The apparatus according to claim 10, wherein, The processor is configured to: Determine the precision required for data processing; and Select the first lookup table and the second lookup table corresponding to the precision from a plurality of pre-generated lookup tables.

15. The apparatus according to claim 14, wherein, When selecting the first lookup table and the second lookup table, the processor is configured to select the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.

16. The apparatus according to claim 10, wherein, The activation function includes the Softmax function.

17. A processor-implemented data processing method for processing division operations included in the activation function of a neural network without any delay caused by the divider, the method comprising: By reading the value of the first lookup table from the memory based on the input data of the activation function, the dividend data corresponding to the dividend of the activation function is determined; By accumulating the dividend data using an accumulator, the divisor data corresponding to the divisor of the activation function is determined; and Output data corresponding to the output of the activation function is determined by reading the values ​​of the second lookup table from the memory based on the dividend data and the divisor data. The determination of the output data includes: Based on the value of the dividend data, determine the column index for addressing the column of the second lookup table; The row index for addressing a row in the second lookup table is determined from the value of the divisor data indicated by the most significant bit of the accumulator; and The value of the second lookup table, addressed according to the column index and the row index, is read from the memory.

18. The method of claim 17, further comprising: The input data is normalized. Determining the dividend data includes: reading the value of the first lookup table addressed according to the normalized input data from the memory.

19. The method of claim 17, wherein, The processor is configured to perform automatic speech recognition based on the determined output data, and The activation function corresponds to the attention function.

20. A processor-implemented data processing method for processing division operations included in the activation function of a neural network without any delay caused by the divider, the method comprising: The input data of the activation function, including the dividend and divisor, is normalized; The dividend data is determined by reading the column index value corresponding to the value of the normalized input data from a first lookup table in one or more memories; The divisor data is determined by accumulating the dividend data using an accumulator. as well as The output data of the activation function is determined by reading the column index corresponding to the value of the normalized input data and the row index corresponding to the value of the divisor data from a second lookup table of the one or more memories. The row index is determined from the value of the divisor data indicated by the most significant bit of the accumulator.

21. The method according to claim 20, wherein, The dividend data is in the range of 0 to 1.

22. The method according to claim 20, wherein, The first lookup table and the second lookup table are determined from multiple generated lookup tables based on the precision of the activation function implemented by the configuration processor.

23. The method according to claim 22, wherein, The precision is any one of floating-point 10, floating-point 32, integer 8, integer 16, and unsigned integer 8.

Images