Methods performed by a hardware accelerator, hardware accelerator, and neural network device

Abstract

A method, a hardware accelerator, and a neural network device are provided for execution using a hardware accelerator. The method includes: receiving an image to be classified; performing computational processing on the image corresponding to a fully connected layer of a neural network to generate result values ​​as probability-related data, which are related to the probability of the image being classified into a corresponding class; loading a lookup table; mapping each probability-related data value to an index in the lookup table based on the probability-related data distribution; using the lookup table to obtain an output data value corresponding to the probability-related data value; and determining the class to which the image belongs based on the output data value. The output data value is proportional to the corresponding flexible maximum value of the probability-related data values.

Description

[0001] This application claims the benefit of U.S. Provisional Application No. 62 / 990,509, filed March 17, 2020, with the United States Patent and Trademark Office, and the benefit of Korean Patent Application No. 10-2020-0114066, filed September 7, 2020, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to methods of execution by hardware accelerators, hardware accelerators, and neural network devices. Background Technology

[0003] Neural networks can be implemented based on computational architectures. Neural network processing devices may require a large amount of computation to process complex input data. Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, the selection of concepts further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a method executed by a hardware accelerator includes: receiving an image to be classified; generating result values, as probability-related data, by performing computational processing corresponding to a fully connected layer of a neural network on the image to be classified; loading a lookup table; mapping each probability-related data value of the probability-related data to an index in the lookup table based on the probability-related data distribution of the probability-related data; obtaining output data values ​​corresponding to the probability-related data values ​​using the lookup table; and determining the class to which the image to be classified belongs based on the output data values, wherein the output data values ​​are proportional to the corresponding flexible maximum value of the probability-related data values.

[0006] The method may further include: calculating the difference between the largest probability-related data value and each probability-related data value.

[0007] The mapping steps may include: directly mapping the difference to an index.

[0008] The mapping steps may include: scaling the difference and mapping the scaled difference to an index.

[0009] The lookup table can store information associated with the reciprocal of the exponential function corresponding to the index.

[0010] Information associated with the reciprocal of the exponential function may include information associated with an integer corresponding to the reciprocal of the exponential function based on the number of bits used for quantization of the output data value.

[0011] The lookup table may include multiple lookup tables, wherein the loading step includes: determining an index range based on the maximum difference among the differences obtained by calculation; and loading one of the multiple lookup tables based on the determined index range.

[0012] The loading steps may include: determining the index range based on the maximum difference among the differences obtained through calculation; and loading the lookup table generated in real time based on the determined index range.

[0013] The steps to obtain the output data value include: mapping the clock of the shift register to an index; and using the shift register to obtain the output data value.

[0014] The method may further include: loading a compensation coefficient lookup table; calculating the sum of output data values ​​corresponding to probability-related data values; mapping the sum of output data values ​​to an index in the compensation coefficient lookup table; using the compensation coefficient lookup table to obtain the compensation coefficients corresponding to the probability-related data values; and obtaining normalized output data values ​​corresponding to the probability-related data values ​​based on the compensation coefficients.

[0015] Hardware accelerators can be neural processors (NPUs).

[0016] The central processing unit (CPU) can be configured to generate lookup tables.

[0017] In another general aspect, a hardware accelerator includes: one or more processors configured to: receive an image to be classified; perform computational processing on the image to be classified corresponding to a fully connected layer of a neural network; generate result values ​​as probability-related data relating to the probability that the image to be classified is assigned to a corresponding class; load a lookup table; map each probability-related data value to an index in the lookup table based on the probability-related data distribution of the probability-related data; use the lookup table to obtain output data values ​​corresponding to the probability-related data values; and determine the class to which the image to be classified belongs based on the output data values, wherein the output data values ​​are proportional to the corresponding flexible maximum value of the probability-related data values.

[0018] The one or more processors may be configured to calculate the difference between the largest probability-related data value and each probability-related data value.

[0019] The one or more processors may be configured to directly map the difference to the index.

[0020] The one or more processors may be configured to scale the difference and map the scaled difference to an index.

[0021] The lookup table can store information associated with the reciprocal of the exponential function corresponding to the index.

[0022] Information associated with the reciprocal of the exponential function may include information associated with an integer corresponding to the reciprocal of the exponential function based on the number of bits used for quantization of the output data value.

[0023] The lookup table may include multiple lookup tables, and the one or more processors may be configured to: determine an index range based on the maximum difference among the differences obtained by calculation; and load one of the multiple lookup tables based on the determined index range.

[0024] The one or more processors may be configured to: determine an index range based on the maximum difference among the differences obtained through calculation; and load a lookup table generated in real time based on the determined index range.

[0025] The one or more processors may be configured to: map the clock of a shift register to an index; and use the shift register to obtain an output data value.

[0026] The one or more processors may be configured to: load a compensation coefficient lookup table; calculate the sum of output data values ​​corresponding to probability-related data values; map the sum of output data values ​​to an index in the compensation coefficient lookup table; use the compensation coefficient lookup table to obtain the compensation coefficients corresponding to the probability-related data values; and obtain normalized output data values ​​corresponding to the probability-related data values ​​based on the compensation coefficients.

[0027] The hardware accelerator may be a neural processor (NPU).

[0028] In another general aspect, a neural network device includes: a central processing unit (CPU) configured to: generate a lookup table storing information associated with the reciprocal of an exponential function; and a neural processing unit (NPU) configured to: receive an image to be classified; perform computational processing on the image to be classified corresponding to a fully connected layer of the neural network; generate result values ​​as probability-related data relating to the probability that the image to be classified is classified into a corresponding class; load the lookup table and obtain output data values ​​corresponding to the probability-related data values; and determine the class to which the image to be classified belongs based on the output data values, wherein the output data values ​​are proportional to the corresponding flexible maximum value of the probability-related data values.

[0029] An NPU can be a hardware accelerator.

[0030] The CPU can also be configured to generate neural networks for image classification.

[0031] The output data values ​​can be obtained as a result of calculating the probability of the image to be classified corresponding to each class.

[0032] Neural networks used for image classification may include loss layers, which are configured to compute a loss as an objective function for learning.

[0033] The NPU can also be configured to scale the difference between the largest probability-related data value and each probability-related data value, and map the scaled difference to an index of the lookup table.

[0034] In one general aspect, a method of operating a hardware accelerator includes: loading a lookup table, mapping each input data value to an index in a plurality of indices in the lookup table based on the input data distribution of the input data; and using the lookup table to obtain an output data value corresponding to the input data value. The output data value is proportional to the corresponding flexible maximum value of the input data value.

[0035] The method may further include: calculating the difference between the maximum input data value and each input data value.

[0036] The mapping steps may include: directly mapping the difference to an index.

[0037] The mapping step may include scaling the difference and mapping the scaled difference to an index.

[0038] The lookup table can store information associated with the reciprocal of the exponential function corresponding to the index.

[0039] Information associated with the reciprocal of the exponential function may include information associated with an integer corresponding to the reciprocal of the exponential function based on the number of bits used for quantization of the output data value.

[0040] The lookup table may include multiple lookup tables. The loading steps may include: determining an index range based on the largest difference among the differences obtained by calculation; and loading one of the multiple lookup tables based on the determined index range.

[0041] The loading steps may include: determining the index range based on the maximum difference among the differences obtained through calculation; and loading the lookup table generated in real time based on the determined index range.

[0042] The steps to obtain the output data value may include: mapping the clock of the shift register to an index; and using the shift register to obtain the output data value.

[0043] The method may further include: loading a compensation coefficient lookup table; calculating the sum of output data values ​​corresponding to the input data values; mapping the sum of the output data values ​​to an index in the compensation coefficient lookup table; using the compensation coefficient lookup table to obtain the compensation coefficient corresponding to the input data values; and obtaining the normalized output data value corresponding to the input data values ​​based on the compensation coefficient.

[0044] A non-transitory computer-readable storage medium stores instructions that, when executed by one or more processors, configure the one or more processors to perform the method.

[0045] In another general aspect, a hardware accelerator includes: one or more processors configured to: load a lookup table, map each input data value to an index in the lookup table based on the input data distribution, and use the lookup table to obtain an output data value corresponding to the input data value. The output data value is proportional to the corresponding flexible maximum value of the input data value.

[0046] The one or more processors may be configured to: calculate the difference between the maximum input data value and each input data value.

[0047] The one or more processors may be configured to directly map the difference to the index.

[0048] The one or more processors may be configured to scale the difference and map the scaled difference to an index.

[0049] The lookup table can store information associated with the reciprocal of the exponential function corresponding to the index.

[0050] Information associated with the reciprocal of the exponential function may include information associated with an integer corresponding to the reciprocal of the exponential function based on the number of bits used for quantization of the output data value.

[0051] The lookup table may include multiple lookup tables. The one or more processors may be configured to: determine an index range based on the maximum difference among the differences obtained through calculation; and load one of the multiple lookup tables based on the determined index range.

[0052] The one or more processors may be configured to: determine an index range based on the maximum difference among the differences obtained through calculation; and load a lookup table generated in real time based on the determined index range.

[0053] The one or more processors may be configured to: map the clock of a shift register to an index; and use the shift register to obtain an output data value.

[0054] The one or more processors may be configured to: load a compensation coefficient lookup table; calculate the sum of output data values ​​corresponding to input data values; map the sum of output data values ​​to an index in the compensation coefficient lookup table; obtain the compensation coefficient corresponding to the input data values ​​using the compensation coefficient lookup table; and obtain the normalized output data values ​​corresponding to the input data values ​​based on the compensation coefficients.

[0055] In another general aspect, a neural network device includes: a central processing unit (CPU) configured to: generate a lookup table storing information associated with the reciprocal of an exponential function; and a neural processing unit (NPU) configured to: load the lookup table and obtain an output data value corresponding to an input data value. The output data value is proportional to the corresponding flexible maximum value of the input data value.

[0056] An NPU can be a hardware accelerator.

[0057] The CPU can also be configured to generate neural networks for classifying input data values.

[0058] The output data value can be obtained as a result of calculating the probability of which class the input data value corresponds to.

[0059] Neural networks used for classification may include loss layers, which are configured to compute a loss as an objective function for learning.

[0060] The NPU can also be configured to scale the difference between the maximum input data value and each input data value, and map the scaled difference to an index of the lookup table.

[0061] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0062] Figure 1 An example of a neural network is shown.

[0063] Figure 2 An example of the hardware configuration of a neural network device is shown.

[0064] Figure 3 An example of a neural network used for classification is shown.

[0065] Figure 4 A flowchart illustrating an example of the flexible maximum approximation method.

[0066] Figure 5 A flowchart illustrating an example of a method that maps each input data value to an index in a lookup table (LUT) based on the difference between the maximum input data value and each input data value.

[0067] Figure 6 A flowchart illustrating an example of a method for scaling the difference between the maximum input data value and each input data value, and mapping the scaled difference to an index in a LUT.

[0068] Figure 7 A flowchart illustrating an example of a method for obtaining output data using multiple LUTs through polynomial approximation.

[0069] Figure 8 A flowchart illustrating an example of a method for obtaining output data using a shift register.

[0070] Figure 9 A flowchart illustrating an example of a method for generating a LUT.

[0071] Figure 10 A flowchart illustrating an example of a method for determining effective quantization boundaries.

[0072] Figure 11 A flowchart illustrating an example of a method for obtaining normalized output data.

[0073] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0074] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various variations, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order of operations set forth herein, but may be changed as will become clear after understanding the disclosure of this application, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known after understanding the disclosure of this application may be omitted.

[0075] The features described herein can be implemented in various forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided merely to illustrate some of the many possible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0076] While terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as a second component, second assembly, second region, second layer, or second part may also be referred to as a second component, second assembly, second region, second layer, or second part.

[0077] Throughout this specification, when a component is described as being "connected to" or "bonded to" another component, that component may be directly "connected to" or "bonded to" said other component, or there may be one or more other elements in between. In contrast, when an element is described as being "directly connected to" or "directly bonded to" another element, there may not be any other elements in between. Similarly, similar expressions (e.g., "between" vs. "immediately between," "adjacent to" vs. "next to") should be interpreted in the same manner. As used herein, the term "and / or" includes any one and any combination of any two or more of the associated listed items.

[0078] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms “comprising,” “including,” and “having” indicate the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0079] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on the understanding of the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having the meaning consistent with their meaning in the context of the relevant field and in the disclosure of this application, and shall not be interpreted in an idealized or overly formalized sense.

[0080] Furthermore, in the description of the exemplary embodiments, such descriptions will be omitted when it is believed that a detailed description of the structure or function known therefrom after understanding the disclosure of this application would lead to a vague interpretation of the exemplary embodiments.

[0081] The following example embodiments can be implemented in various forms of products, such as personal computers (PCs), laptop computers, tablet PCs, smartphones, televisions (TVs), smart home appliances, smart vehicles, self-service kiosks, and wearable devices. Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings, and the same reference numerals in the drawings always denote the same elements.

[0082] The desired approach is one that efficiently handles computations or operations involving neural networks, enabling the analysis of large amounts of input data in real time and the extraction of desired information using neural networks.

[0083] Figure 1 An example of a neural network is shown.

[0084] exist Figure 1 The example illustrates a neural network 10. The neural network 10 may have an architecture including an input layer, hidden layers, and an output layer, and may be configured to perform computations or operations based on received input data (e.g., I1 and I2), and generate output data (e.g., O1 and O2) based on the results of the computations. It should be noted here that the term "may" (e.g., what may or may be implemented in an example or embodiment) is used with respect to an example or embodiment to indicate the existence of at least one example or embodiment that includes or implements such a feature, but all examples and embodiments are not limited thereto.

[0085] Neural network 10 can be a deep neural network (DNN) including one or more hidden layers or an n-layer neural network. For example, Figure 1 As shown, neural network 10 can be a DNN comprising an input layer (layer 1), two hidden layers (layer 2 and layer 3), and an output layer (layer 4). DNNs can include, for example, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), and restricted Boltzmann machines (RBMs), but the examples are not limited to these.

[0086] In an example of a DNN architecture in neural network 10, neural network 10 may include a larger number of layers that extract available information, thereby handling more complex datasets compared to existing neural networks. Although in Figure 1 The neural network 10 is shown as comprising four layers, but the example is not limited to this; the neural network 10 may include a number of layers less than or greater than the four shown. Furthermore, the neural network 10 may include layers of various architectures different from the one shown. For example, the neural network 10 may be a DNN comprising convolutional layers, pooling layers, and fully connected (FC) layers.

[0087] Each layer included in the neural network 10 may include multiple artificial nodes, each known by terms such as "neuron," "processing element (PE)," and "unit." For example, as Figure 1 As shown, layer 1 may include two nodes, and layer 2 may include three nodes. However, the number of nodes is provided only as an example, and each layer included in the neural network 10 may include a variety of numbers of nodes.

[0088] In the neural network 10, the nodes in the layers can be connected to each other to exchange data. For example, a node can receive data from other nodes, perform computations on the received data, and output the computation results to other nodes.

[0089] The output value of each node can be referred to here as its activation. An activation can be the output value of a node and the input values ​​of nodes included in subsequent layers. Each node can determine its own activation based on the activations and weights received from nodes included in previous layers. Weights can represent parameters used to calculate the activation at each node and are assigned to the connections between nodes.

[0090] Each node can be, for example, a hardware computing unit that receives input activation and output activation and performs input-output mapping. For example, where σ represents the activation function, This represents the weights from the k-th node included in layer (i-1) to the j-th node included in layer i. This represents the bias value of the j-th node included in the i-th layer, and In the case of the activation of the j-th node in the i-th layer, the activation is... It can be represented by Equation 1.

[0091] Equation 1:

[0092]

[0093] For example, such as Figure 1 As shown, the activation of the first node in the second layer (layer 2) can be represented as In this example, based on Equation 1 above, activation It can have the following characteristics: The value is represented by Equation 1. Equation 1 is provided only as an example to describe the activations and weights used to process data in a neural network, and therefore the example is not limited thereto. For example, an activation could be a value obtained by applying a value obtained by applying an activation function to a weighted sum of activations received from the previous layer to a linear rectified function (ReLU, also known as a modified linear unit).

[0094] As mentioned above, in neural networks, many datasets can be exchanged between multiple interconnected nodes and undergo a great deal of computational processing as they pass through layers. Therefore, a method is desired that minimizes accuracy loss while reducing the amount of computation required to process complex input data.

[0095] Figure 2 An example of the hardware configuration of a neural network device is shown.

[0096] exist Figure 2 In this context, as a non-limiting example, the neural network device 200 may include a host 210, a hardware accelerator 230, and a memory 220.

[0097] The neural network device 200 can be a computing device with various processing functions (such as generating neural networks, training or learning neural networks, quantizing floating-point neural networks into fixed-point neural networks, and retraining neural networks). For example, the neural network device 200 can be various types of devices (e.g., PCs, remote server devices, mobile devices, etc.) or can be implemented by various types of devices (e.g., PCs, remote server devices, mobile devices, etc.).

[0098] The host 210 can perform overall functions for controlling the neural network device 200. For example, the host 210 can control the neural network device 200 as a whole by executing instructions stored in the memory 220 of the neural network device 200. The host 210 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), and an application processor (AP) included in the neural network device 200, or may be implemented by, for example, a central processing unit (CPU), a graphics processing unit (GPU), and an application processor (AP) included in the neural network device 200, but the examples are not limited thereto.

[0099] Host 210 can generate and train a neural network for classification. The neural network for classification can output a computational result obtained by calculating which class the input data belongs to. For example, the neural network for classification can output a computational result obtained by calculating the probability of the input data corresponding to each class as a result value for each class. The neural network for classification may include a softmax layer and a loss layer. The softmax layer can convert the result value for each class into a probability value, and the loss layer can calculate the loss as an objective function for training or learning.

[0100] Memory 220 may be hardware configured to store various datasets, and may store datasets that are processed or will be processed in the neural network device 200. Furthermore, memory 220 may store applications, drivers, etc., that will be executed or driven by the neural network device 200. Memory 220 may be dynamic random access memory (DRAM), but is not limited thereto. Memory 220 may include at least one of volatile memory and non-volatile memory.

[0101] The neural network device 200 may also include a hardware accelerator 230 for operating the neural network. The hardware accelerator 230 may be a module dedicated to operating the neural network and may include, for example, a neural processor (NPU), a tensor processor (TPU), and a neural engine, but is not limited to these examples.

[0102] Figure 3 An example of a neural network used for classification is shown.

[0103] exist Figure 3In the neural network 300 used for classification, there may be a hidden layer 310, a fully connected (FC) layer 320, a maximum flexibility layer 330, and a loss layer 340. A portion of the hidden layer 310 may be an FC layer, therefore the FC layer 320 may be the last FC layer in the neural network 300. That is, the FC layer 320 may be the last FC layer to appear sequentially among the FC layers of the neural network 300.

[0104] When input data is fed into neural network 300, sequential computation is performed through hidden layer 310 and FC layer 320. The computation result s corresponding to the probability of the input data being classified into each class can then be output from FC layer 320. In other words, FC layer 320 can output the result value corresponding to the probability of the input data being classified into the corresponding class as the computation result s (or probability-related data) for each class. In one example, when an image to be classified is input into neural network 300, computation can be performed on the image through hidden layer 310 and FC layer 320 to generate result values ​​related to the probability of the image being classified into the corresponding class, which serve as input data for the maximum flexible layer 330 of neural network 300. For example, FC layer 320 may include nodes corresponding to multiple classes, and each node of FC layer 320 can output the result value corresponding to the probability of the input data being classified into the corresponding class. For example, in the case where a neural network is executed to classify into five classes, the output value of each of the first to fifth nodes of the FC layer of the neural network can be a result value indicating the probability that the input data is classified into each of the first to fifth classes.

[0105] The FC layer 320 outputs the computation result s to the flexible maximization layer 330, which then converts the computation result s into a probability value y. Specifically, the flexible maximization layer 330 generates the probability value y by normalizing the result value corresponding to the probability that the input data is classified into each class. The flexible maximization layer 330 then outputs the probability value y to the loss layer 340, which calculates the cross-entropy loss L of the computation result s based on the probability value y. In other words, the loss layer 340 calculates the cross-entropy loss L, which indicates the error in the computation result s.

[0106] For example, the flexible maximum layer 330 can use the flexible maximum operation as shown in Equation 2 to convert the calculation result s into a probability value y, and the loss layer 340 can use Equation 3 as shown below to calculate the cross-entropy loss L of the calculation result s.

[0107] Equation 2:

[0108]

[0109] Equation 3:

[0110]

[0111] In equations 2 and 3, s i This represents the output value of the i-th node in FC layer 320 (e.g., the result value for the i-th class in the class). i This represents the output value of the i-th node in the maximum flexible layer 330 (e.g., the probability value of the i-th class in the class). N c Indicates the number of classes. t i It can be the ground truth label of class i.

[0112] Subsequently, backpropagation learning can be performed. The gradient of the cross-entropy loss L can be calculated through loss layer 340 and flexible maximum layer 330.

[0113] For example, the gradient of the cross-entropy loss L in the flexible maximum layer 330 can be calculated using Equation 4 below (e.g., ).

[0114] Equation 4:

[0115]

[0116] In equation 4, s i This represents the output value of the i-th node in FC layer 320 (e.g., the result value for the i-th class in the class). i This represents the output value of the i-th node in the maximum flexible layer 330 (e.g., the probability value of the i-th class in the class). N c Indicates the number of classes. t i This represents the GT tag of class i.

[0117] Subsequently, gradient learning based on the cross-entropy loss L can be performed in the FC layer 320. For example, the weights of the FC layer 320 can be updated according to the gradient descent algorithm. Furthermore, subsequent learning processing can be performed in the hidden layer 310.

[0118] Return to reference Figure 2 The hardware accelerator 230 can perform flexible maximization. Typical hardware accelerators do not perform complex flexible maximization, but only matrix-vector multiplication. Therefore, a typical neural network device can perform matrix-vector multiplication only through the hardware accelerator, move the result of the multiplication to the host computer, perform flexible maximization on the host computer, and then move the result of the flexible maximization back to the hardware accelerator. This typical approach may make the development of software for data transformation and movement more complex, which could adversely affect overall performance and power consumption.

[0119] For flexible maximum computation, a method has been proposed to perform flexible maximum computation using hardware accelerators to prevent host intervention; however, this method may require a divider, which may result in additional hardware costs.

[0120] According to the example embodiment, hardware accelerator 230 can perform flexible maximum operation without such a divider. Hardware accelerator 230 can convert probability-related data to have positive values ​​while preventing overflow, and achieve and perform efficient flexible maximum approximation by directly using the value obtained through the conversion as an index to read the appropriate value through a lookup table (LUT).

[0121] The hardware accelerator 230 can estimate the maximum flexibility value through an approximation operation represented by Equation 5 below, rather than the maximum flexibility operation represented by Equation 2 above.

[0122] Equation 5:

[0123]

[0124] In Equation 5, x represents each input data value (i.e., probability-related data) of the maximum flexible layer, and y represents the output data of the maximum flexible layer corresponding to each probability-related data value. max(x) represents the maximum probability-related data value. Therefore, max(x) - x represents the difference between the maximum probability-related data value and each probability-related data value.

[0125] In Equation 2 above, the maximum flexibility value can be the probability value of the exponential function of each probability-related data value x. However, in Equation 5, the output data can be the exponential function value of the probability-related data for each transformation. Therefore, the output data y in Equation 5 can be proportional to the maximum flexibility value of each probability-related data value x.

[0126] In Equation 5, the hardware accelerator 230 can convert probability-related data into positive data by reverse calculation (e.g., max(x)-x, instead of x-max(x)) and perform a reciprocal transformation (e.g., 1 / exp(x)) to compensate for such reverse calculation.

[0127] Furthermore, although described below, hardware accelerator 230 can convert probabilistic data to positive values ​​while preventing overflow, and map the values ​​obtained through the transformation to indices in the LUT. However, in examples using probabilistic data without such a transformation, the range of the probabilistic data may not be defined, making it potentially difficult to construct a LUT corresponding to the probabilistic data. Additionally, in examples where probabilistic data is converted to negative values ​​via forward computation (e.g., x-max(x)), additional transformations or alterations (e.g., scaling and / or biasing) may be required to map the indices in the LUT, potentially leading to overload.

[0128] According to the example embodiment, by using only a 1D LUT to compute the flexible maximum value, the size of the LUT can be reduced, the computational complexity can be greatly reduced without the need for dividers and / or multipliers, and it can be implemented in a hardware accelerator 230 (such as an NPU) without requiring large hardware resources.

[0129] Figure 4 A flowchart illustrating an example of the flexible maximum approximation method is shown below. (Refer to the following text.) Figure 4 Operations 410 to 430 described herein may be performed by a hardware accelerator (e.g., Figure 2 The hardware accelerator 230) executes the operation. The hardware accelerator may be one or more hardware modules, one or more software modules, or various combinations thereof, or may be implemented by one or more hardware modules, one or more software modules, or various combinations thereof.

[0130] exist Figure 4 In operation 410, the hardware accelerator loads the LUT. The hardware accelerator can load LUTs stored in the host or LUTs generated in real time in the host. For example, the NPU can load LUTs generated in the CPU.

[0131] A LUT can store information associated with the reciprocal of an exponential function corresponding to a preset range. For example, a LUT can include information associated with values ​​from 1 / exp(0) to 1 / exp(n), where n represents a positive integer. A LUT can indicate an approximate value of the reciprocal of an exponential function, as shown in Table 1 below.

[0132] Table 1:

[0133] Index (id) LUT(id) 0 1.000 1 0.368 2 0.135 3 0.050 4 0.018 5 0.007 6 0.002 7 0.001

[0134] Furthermore, the information associated with the reciprocal of the exponential function may include information associated with integers that correspond to the reciprocal of the exponential function based on the number of bits used for quantization of the output data. For example, the output data may be quantized to [0, 255] using 8 bits, and the LUT may include information associated with integers within [0, 255] that are proportional to the reciprocal of the same exponential function, as shown in Table 2 below.

[0135] Table 2:

[0136] Index (id) LUT(id) 0 255 1 94 2 35 3 13 4 5 5 2 6 1 7 0

[0137] Similarly, the output data can be quantized to [0,15] using 4 bits, and the LUT can include information associated with integers in [0,15] that are proportional to the reciprocal of the same exponential function, as shown in Table 3 below.

[0138] Table 3:

[0139] Index (id) LUT(id) 0 15 1 6 2 2 3 1 4 0

[0140] In operation 420, the hardware accelerator maps each probability-related data value to an index of the LUT based on the probability-related data distribution. In one example, the probability-related data distribution could be the difference between the largest probability-related data value and each probability-related data value, but this application is not limited to this. Here, the probability-related data could be the data to be input into the flexible maximum layer (e.g., as shown above). Figure 3 The computational results described are as follows: The hardware accelerator can calculate the difference between the largest probability-related data value and each probability-related data value, and map each probability-related data value to an index in the LUT based on the calculated difference.

[0141] In one example, the hardware accelerator can directly map the largest probability-related data value to the difference between each probability-related data value and an index in the LUT. For instance, if the probability-related data are 990, 991, 992, 992, 990, 995, 993, and 997, the hardware accelerator can directly map the largest probability-related data value (e.g., 997) to the difference between each probability-related data value (e.g., 7, 6, 5, 5, 7, 2, 4, and 0) to an index in the LUT without any processing.

[0142] Furthermore, in a non-limiting example, the host may store multiple LUTs, and the hardware accelerator may load one of the multiple LUTs. The hardware accelerator may determine the LUT index range based on the maximum difference between the calculated largest probability-related data value and the differences between each probability-related data value, and load one of the LUTs based on the determined index range.

[0143] For example, if the LUTs indicated in Tables 2 and 3 above are stored in the host and the probability-related data are 990, 991, 992, 992, 990, 995, 993, and 997, the hardware accelerator can determine the index range as [0, 7] based on the largest difference of 7 among the differences between the largest probability-related data value 997 and each probability-related data value (7, 6, 5, 5, 7, 2, 4, and 0), and then load the LUT from Table 2 corresponding to the determined index range as the final LUT. In this example, when loading one of the LUTs, the hardware accelerator can load the LUT after calculating the difference between the largest probability-related data value and each probability-related data value.

[0144] In operation 430, the hardware accelerator uses a LUT to obtain output data corresponding to each probability-related data value. (See above reference...) Figure 3 The output data can be data output from the flexible maximum layer (e.g., relative probability values ​​indicating the probability that the input data of the neural network is classified into each class).

[0145] For example, if the input data (or probability-related data) of the maximum flexible layer are 990, 991, 992, 992, 990, 995, 993, and 997, the hardware accelerator can output LUT(7), LUT(6), LUT(5), LUT(5), LUT(7), LUT(2), LUT(4), and LUT(0) as output data corresponding to each probability-related data value. In the LUTs in Table 1 above, the output data can be 0.001, 0.002, 0.007, 0.007, 0.001, 0.135, 0.018, and 1.0. In this example, the input data of the neural network can be classified into class 8 corresponding to the maximum output data value (e.g., 1).

[0146] Here, 2n loops are sufficient for the hardware accelerator to perform the most flexible computation. For example, n loops might be needed to obtain the maximum probability-related data value, and n loops might be needed to load the LUT and obtain the output data. In the following text, references will be made to... Figures 5 to 7 This section provides a detailed example of mapping each probability-related data value to an index in a LUT based on the probability-related data distribution.

[0147] Figure 5 A flowchart illustrating an example of a method that maps each probability-related data value to an index in a LUT based on the difference between the largest probability-related data value and each other probability-related data value.

[0148] exist Figure 5 In this context, hardware accelerators can load LUTs with an index range of [0, x_q] from the host (e.g., CPU).

[0149] The hardware accelerator can extract the largest probability-related data value and calculate the difference between the extracted largest probability-related data value and each probability-related data value.

[0150] Hardware accelerators can directly map the difference between the largest probability-related data value and each probability-related data value to an index in the LUT.

[0151] Hardware accelerators can use LUTs to obtain output data corresponding to each probability-related data value. As mentioned above, the output data can indicate a relative probability value, which indicates the probability that the input data of the neural network is classified into each class.

[0152] The hardware accelerator can directly map the difference between the largest probability-related data value and each probability-related data value to an index in the LUT, and obtain a relative probability value indicating the probability that the input data of the neural network is classified into each class, without the need for a divider.

[0153] Figure 6 The flowchart illustrates an example of a method that scales the difference between the largest probability-related data value and each probability-related data value and maps the scaled difference to an index in a LUT.

[0154] exist Figure 6 In this process, the hardware accelerator can scale the difference between the largest probability-related data value and each probability-related data value (e.g., x), and map the scaled difference to an index in the LUT.

[0155] For example, when the difference between the largest probability-related data value and each probability-related data value has a range of [0, 10] and the index of the loaded LUT has a range of [0, 20], scaling the difference to fit the index range can more effectively improve the accuracy of the output data than mapping the difference directly to the index.

[0156] The hardware accelerator can extract the largest probability-related data value and calculate the difference between the largest probability-related data value and each probability-related data value.

[0157] The hardware accelerator can scale the difference between the largest probability-related data value and each probability-related data value, and then map the scaled difference to an index. As an example, in... Figure 6 In this context, based on the equation id x = ax + b, the difference between the largest probability-related data value and each other probability-related data value can be scaled to fit the index range, where a and b are scaling factors, and id x is the scaled difference. In the previous example, based on the equation id x = 2x, the hardware accelerator can map the difference between the largest probability-related data value and each other probability-related data value to the index.

[0158] Hardware accelerators can use LUTs (e.g., via y = LUT[idx]) to obtain output data corresponding to each probability-related data value.

[0159] Figure 7 A flowchart illustrating an example of a method for obtaining output data using multiple LUTs through polynomial approximation.

[0160] exist Figure 7 In this context, hardware accelerators can use multiple LUTs to obtain output data through polynomial approximation.

[0161] The hardware accelerator can flexibly control the approximation accuracy based on the selected polynomial. As an example, the hardware accelerator can load multiple LUTs (or families of LUTs), such as LUT_a, LUT_b, LUT_c, etc. For instance, the hardware accelerator can approximate using a case-based approach from a first-order polynomial approximation method 710 (e.g., y = LUT_a[idx]*x + LUT_b[idx]) and a second-order polynomial approximation method 720 (e.g., y = LUT_a[idx]*x). 2 Choose one of LUT_b[idx]*x+LUT_c[idx]) to flexibly control the approximate accuracy.

[0162] Figure 8 A flowchart illustrating an example of a method for obtaining output data using a shift register.

[0163] exist Figure 8 In this process, the hardware accelerator can map the clock of the shift register to the index and use the shift register to obtain the output data.

[0164] The hardware accelerator can extract the largest probability-related data value and calculate the difference between the extracted largest probability-related data value and each probability-related data value.

[0165] The hardware accelerator can map the clock of the shift register to an index based on the difference between the calculated maximum probability-related data value and each probability-related data value.

[0166] Hardware accelerators can use shift registers to obtain output data, which corresponds to the clock signal between the largest probability-related data value and the difference between each probability-related data value. For example, using a 4-bit shift register, the hardware accelerator can obtain the outputs indicated in Table 4 below.

[0167] Table 4:

[0168] Index (id) LUT(id) 0 <![CDATA[15(1111 (2) )]]> 1 <![CDATA[7(0111 (2) )]]> 2 <![CDATA[3(0011 (2) )]]> 3 <![CDATA[1(0001 (2) )]]> 4 <![CDATA[0(0000 (2) )]]>

[0169] Figure 9 A flowchart illustrating an example of a method for generating a LUT.

[0170] exist Figure 9 In this scenario, the host (e.g., CPU) can extract the largest probability-related data value and calculate the difference between the extracted largest probability-related data value and each other probability-related data value. Alternatively, the host can receive the difference between the largest probability-related data value and each other probability-related data value from a hardware accelerator (e.g., NPU) instead of calculating the difference directly.

[0171] The host can extract the largest difference among the differences obtained by calculating the largest probability-related data value and the differences between each probability-related data value. For example, when the probability-related data are 990, 991, 992, 992, 990, 995, 993, and 997, the host can determine the largest difference among the differences between the largest probability-related data value 997 and the individual probability-related data values ​​of 7, 6, 5, 5, 7, 2, 4, and 0 as 7.

[0172] The host can generate a LUT based on the maximum difference and the number of bits used for quantization of the output data. In the previous example, the host can determine the LUT index range based on the maximum difference of 7, and generate information associated with an integer that is proportional to the reciprocal of the exponential function corresponding to the index, based on the number of bits used for quantization of the output data. For example, the host can generate the LUT indicated in Table 2 above based on the maximum difference of 7 and the number of bits of 8.

[0173] Figure 10 A flowchart illustrating an example of a method for determining effective quantization boundaries.

[0174] exist Figure 10 In this context, the hardware accelerator can determine the effective quantization boundary as shown in Equation 6.

[0175] Equation 6:

[0176] x_q=ln(2 w -1)

[0177] In Equation 6, w represents the number of bits used for quantization of the output data. For example, w could be determined as 7 when using an int8 hardware accelerator. For example, w could be determined as 3 when using an int4 hardware accelerator. For example, w could be determined as 4 when using a uint4 hardware accelerator.

[0178] Hardware accelerators can determine the quantization boundaries based on Equation 6 above and efficiently utilize the available quantization range based on those boundaries. That is, in such an example, the LUT can provide a higher level of accuracy with the same number of quantization bits. When determining the quantization boundaries based on Equation 6 above, it may be necessary to recalculate the LUT's contents based on x_q.

[0179] Figure 11 A flowchart illustrating an example of a method for obtaining normalized output data.

[0180] The above reference Figures 1 to 10 The described hardware accelerator operation can be applied to Figure 11 Examples are provided, so more detailed and repetitive descriptions will be omitted here.

[0181] The hardware accelerator can estimate the maximum flexibility using the approximation represented by Equation 5 above. However, in examples of performing complex tasks using the approximate maximum flexibility in Equation 5, the final accuracy of the task may degrade to a level that disrupts the task's normal performance due to approximation errors. The hardware accelerator can obtain normalized output data by applying a compensation coefficient α to the approximate maximum flexibility. For example, the approximate maximum flexibility could be a scaled version of the actual maximum flexibility, so the hardware accelerator can obtain normalized output data by assigning a scaling factor to the approximate maximum flexibility. Normalized output data can be obtained as shown in Equation 7.

[0182] Equation 7:

[0183] y(x)=α*y ～ (x)

[0184] In equation 7, y ～ y(x) represents the maximum flexibility value estimated by Equation 5 above, α represents the compensation coefficient, and y(x) represents the normalized output data. The compensation coefficient α can be calculated as shown in Equation 8.

[0185] Equation 8:

[0186]

[0187] In equation 8, y ～ This represents the approximate maximum flexibility. The hardware accelerator can calculate the sum of the approximate maximum flexibility (e.g., sum_rexp) and map the sum to an index of the compensation coefficient α (e.g., alpha_idx). The hardware accelerator can use a preset function to map the sum of the approximate maximum flexibility to an index of the compensation coefficient α. For example, the hardware accelerator can map the sum of the approximate maximum flexibility to an index of the compensation coefficient α as shown in Equation 9.

[0188] Equation 9:

[0189] alpha_idx = round(sum_rexp)

[0190] Equation 9 is provided only as an example of mapping the sum of approximate maximum flexibility values ​​to an index of the compensation coefficient α. As a non-restrictive example, various functions (f(sum_rexp)) (e.g., alpha_idx = round(sum_rexp)-1, alpha_idx = ceil(sum_rexp)-1, alpha_idx = floor(sum_rexp) etc.) can be used to perform the mapping.

[0191] The hardware accelerator can use a compensation coefficient LUT (e.g., LUT_expln) that includes information associated with the compensation coefficient α (e.g., alpha) to obtain the compensation coefficient α corresponding to the index, and obtain the normalized output data corresponding to the probability-related data by multiplying each approximate maximum flexibility value by the compensation coefficient α.

[0192] For example, Table 5 below indicates the compensation coefficient LUT corresponding to 16 indices using signed 16 bits.

[0193] Table 5:

[0194]

[0195]

[0196] For example, Table 6 below indicates the compensation coefficient LUT corresponding to 16 indices using unsigned 8 bits.

[0197] Table 6:

[0198]

[0199]

[0200] For example, Table 7 below indicates the compensation coefficient LUT corresponding to 16 indices using signed 4 bits.

[0201] Table 7:

[0202] Index (id) LUT_expln(id) 0 15 1 8 2 5 3 4 4 3 5 3 6 2 7 2 8 2 9 2 10 1 11 1 12 1 13 1 14 1 15 0

[0203] For example, Table 8 below indicates the compensation coefficient LUT corresponding to the seven indices using signed 2 bits.

[0204] Table 8:

[0205] Index (id) LUT_expln(id) 0 3 1 2 2 1 3 1 4 1 5 1 6 0

[0206] Furthermore, to improve accuracy, the scaling factor of the compensation coefficient LUT can be adjusted. For example, when the compensation coefficient LUT is scaled to LUT_expln[0, scale*sum_q], the index of the compensation coefficient α can be determined based on alpha_idx = round(scale*sum_rexp) - 1, where [0, sum_q] represents the range of the original compensation coefficient LUT, and scale represents the scaling factor.

[0207] In this regard Figures 1 to 11The described neural network device, host, hardware accelerator, memory, neural network device 200, host 210, hardware accelerator 230, memory 220, and other devices, apparatuses, units, modules, and other components are implemented via hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components performing the operations described in this application are implemented via computing hardware (e.g., via one or more processors or computers). A processor or computer can be implemented via one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For the sake of brevity, the singular terms "processor" or "computer" are used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0208] Figures 1 to 11The methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., by one or more processors or a computer), which is implemented as described above to execute instructions or software to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations may be executed by a single processor or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation or two or more operations.

[0209] Instructions or software for controlling a processor or computer to implement hardware components and perform the methods described above are written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure the processor or computer to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by the processor or computer. In another example, the instructions or software include high-level code that is executed by the processor or computer using an interpreter. Programmers of ordinary skill in the art can readily write instructions or software based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0210] Instructions or software used to control a processor or computer to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, are recorded, stored, or fixed in, or on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), programmable random access read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store instructions or software and any associated data, data files and data structures in a non-transitory manner and to provide instructions or software and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the instructions.

[0211] While this disclosure includes specific examples, it will be apparent to those skilled in the art that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example will be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

[0212] Therefore, the scope of the disclosure is not limited by the specific embodiments, but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be interpreted as included in the disclosure.

Claims

1. A method for execution by a hardware accelerator, comprising: Receive the image to be classified; By performing computational processing corresponding to the fully connected layer of the neural network on the image to be classified, a result value related to the probability that the image to be classified is classified into the corresponding class is generated as probability-related data. Load the lookup table; Based on the probability-related data distribution, each probability-related data value is mapped to an index in the lookup table; Use a lookup table to obtain the output data values ​​corresponding to the probability-related data values; and The class to which the image to be classified belongs is determined based on the output data values. Among them, the output data value is proportional to the corresponding maximum flexibility of the probability-related data value. The method further includes: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the probability-related data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficients corresponding to the probability-related data values; and The normalized output data values ​​corresponding to the probability-related data values ​​are obtained based on the compensation coefficient.

2. The method according to claim 1, further comprising: Calculate the difference between the largest probability-related data value and each other probability-related data value.

3. The method according to claim 2, wherein, The mapping steps include: The difference is directly mapped to the index.

4. The method according to claim 2, wherein, The mapping steps include: The difference is scaled and the scaled difference is mapped to an index.

5. The method according to claim 1, wherein, The lookup table stores information associated with the reciprocal of the exponential function corresponding to the index.

6. The method according to claim 5, wherein, Information associated with the reciprocal of the exponential function includes information associated with an integer corresponding to the reciprocal of the exponential function, based on the number of bits used for quantization of the output data value.

7. The method according to claim 2, wherein, The lookup table includes multiple lookup tables, and The loading steps include: The index range is determined based on the maximum difference among the differences obtained through calculation; and One of the multiple lookup tables is loaded based on a defined index range.

8. The method according to claim 2, wherein, The loading steps include: The index range is determined based on the maximum difference among the differences obtained through calculation; and Load the lookup table generated in real time based on a defined index range.

9. The method according to claim 1, wherein, The steps to obtain the output data value include: Map the clock of the shift register to the index; and Use a shift register to obtain the output data value.

10. The method according to any one of claims 1 to 9, wherein, Hardware accelerators are neural processors.

11. The method according to any one of claims 1 to 9, wherein, The central processing unit is configured to generate lookup tables.

12. A non-transitory computer-readable storage medium storing instructions, which, when executed by one or more processors, configure the one or more processors to perform the method according to any one of claims 1 to 11.

13. A hardware accelerator, comprising: One or more processors are configured to: receive an image to be classified; perform computational processing on the image to be classified corresponding to a fully connected layer of a neural network; generate result values ​​as probability-related data that correlate with the probability of the image being classified into a corresponding class; load a lookup table; map each probability-related data value to an index in the lookup table based on the probability-related data distribution; use the lookup table to obtain output data values ​​corresponding to the probability-related data values; and determine the class to which the image to be classified belongs based on the output data values. Among them, the output data value is proportional to the corresponding maximum flexibility of the probability-related data value. Wherein, the one or more processors are configured as follows: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the probability-related data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficients corresponding to the probability-related data values; and The normalized output data values ​​corresponding to the probability-related data values ​​are obtained based on the compensation coefficient.

14. The hardware accelerator according to claim 13, wherein, The one or more processors are configured to: Calculate the difference between the largest probability-related data value and each other probability-related data value.

15. The hardware accelerator according to claim 14, wherein, The one or more processors are configured to: The difference is directly mapped to the index.

16. The hardware accelerator according to claim 14, wherein, The one or more processors are configured to: The difference is scaled and the scaled difference is mapped to an index.

17. The hardware accelerator according to claim 13, wherein, The lookup table stores information associated with the reciprocal of the exponential function corresponding to the index.

18. The hardware accelerator according to claim 17, wherein, Information associated with the reciprocal of the exponential function includes information associated with an integer corresponding to the reciprocal of the exponential function, based on the number of bits used for quantization of the output data value.

19. The hardware accelerator according to claim 14, wherein, The lookup table includes multiple lookup tables. Wherein, the one or more processors are configured as follows: The index range is determined based on the maximum difference among the differences obtained through calculation; and One of the multiple lookup tables is loaded based on a defined index range.

20. The hardware accelerator of claim 14, wherein, The one or more processors are configured to: The index range is determined based on the maximum difference among the differences obtained through calculation; and Load the lookup table generated in real time based on a defined index range.

21. The hardware accelerator according to claim 13, wherein, The one or more processors are configured to: Map the clock of the shift register to the index; and Use a shift register to obtain the output data value.

22. The hardware accelerator according to any one of claims 13 to 21, wherein, The hardware accelerator is a neural processor.

23. A neural network device, comprising: The central processing unit is configured to generate a lookup table that stores information associated with the reciprocal of the exponential function; as well as The neural processor is configured to: receive an image to be classified; perform computational processing on the image corresponding to the fully connected layer of the neural network; generate result values ​​related to the probability of the image being classified into the corresponding class as probability-related data; load a lookup table and obtain output data values ​​corresponding to the probability-related data values; and determine the class to which the image belongs based on the output data values. Among them, the output data value is proportional to the corresponding maximum flexibility of the probability-related data value. The neural processor is configured as follows: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the probability-related data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficients corresponding to the probability-related data values; and The normalized output data values ​​corresponding to the probability-related data values ​​are obtained based on the compensation coefficient.

24. The apparatus according to claim 23, wherein, Neural processors are hardware accelerators.

25. The apparatus according to claim 23, wherein, The central processing unit is also configured to generate quantized neural networks for deep neural network models.

26. The apparatus according to claim 25, wherein, The output data values ​​are obtained as a result of calculating the probability of the image to be classified corresponding to each class.

27. The apparatus according to claim 25, wherein, The quantization neural network used for deep neural network models includes a loss layer, which is configured to compute the loss as an objective function for learning.

28. The apparatus according to claim 23, wherein, The neural processor is also configured to scale the difference between the largest probability-related data value and each probability-related data value, and to map the scaled difference to an index of a lookup table.

29. A method of operating a hardware accelerator, comprising: Load the lookup table from the host; Based on the distribution of the input data, each input data value is mapped to an index in the lookup table; as well as Use a lookup table to obtain the output data value corresponding to the input data value. The output data value is proportional to the corresponding maximum flexibility of the input data value. The method further includes: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the input data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficient corresponding to the input data value; and The normalized output data value corresponding to the input data value is obtained based on the compensation coefficient.

30. A hardware accelerator, comprising: One or more processors are configured to: load a lookup table from the host, map each input data value to an index in the lookup table based on the input data distribution, and use the lookup table to obtain the output data value corresponding to the input data value. The output data value is proportional to the corresponding maximum flexibility of the input data value. Wherein, the one or more processors are configured as follows: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the input data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficient corresponding to the input data value; and The normalized output data value corresponding to the input data value is obtained based on the compensation coefficient.

31. A neural network device, comprising: The central processing unit is configured to generate a lookup table that stores information associated with the reciprocal of the exponential function; as well as The neural processor is configured to: load a lookup table and obtain the output data value corresponding to the input data value. The output data value is proportional to the corresponding maximum flexibility of the input data value. The neural processor is configured as follows: Load the compensation coefficient lookup table; Calculate the sum of the output data values ​​corresponding to the input data values; Map the sum of the output data values ​​to an index in the compensation coefficient lookup table; Use the compensation coefficient lookup table to obtain the compensation coefficient corresponding to the input data value; and The normalized output data value corresponding to the input data value is obtained based on the compensation coefficient.

Images