Arithmetic device, system and operating method thereof

Abstract

An arithmetic device, system and operating method thereof are provided. The arithmetic device includes: a separator circuit for separating an input data value into an integer portion and a decimal portion; a compensation circuit for generating a compensated decimal portion according to at least a first output of the first comparator circuit comparing the decimal portion with a first threshold; a scheduler circuit for dynamically determining the terms of the exponent for approximating the fractional part; and the Taylor expansion calculation circuit is used for calculating the sum of Taylor expansion terms of the compensation decimal part.

Description

Technical Field

[0001] The embodiments disclosed herein relate to a computing device and its operation method, and more particularly to a computing device and its operation method for an arithmetic logic unit for a computing processor. Background Technology

[0002] Artificial intelligence (AI) systems can be built on software-based neural network models implemented on one or more AI accelerators, such as graphics processing units (GPUs) and tensor processing units (TPUs). AI accelerators can contain dedicated hardware components and / or devices to accelerate the execution of AI and machine learning workloads. Existing AI accelerators and / or processors largely rely on software frameworks and programs to perform complex computational tasks. Due to the high computational demands of AI systems, the power consumption of such AI accelerators and / or processors can be significant. Summary of the Invention

[0003] One embodiment of this disclosure provides a computing device for performing input data calculations in a neural network. The computing device includes: a separator circuit for separating an input data value into an integer part and a fractional part; a compensation circuit for generating a compensated fractional part based on at least one first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold; a scheduler circuit for dynamically determining the number of terms used to approximate the fractional part; and a Taylor expansion calculation circuit for calculating the sum of the Taylor expansion terms of the compensated fractional part.

[0004] The embodiments disclosed herein provide an operation method for a computing device. The operation method includes: separating an input data value into an integer part and a fractional part using a separator circuit; generating a compensated fractional part using a compensation circuit based on at least one first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold; dynamically determining the number of terms for the exponent used to approximate the fractional part using a scheduler circuit; calculating the sum of the Taylor expansion terms of the compensated fractional part as an approximate exponent of the compensated fractional part using a Taylor expansion calculation circuit; and outputting the exponent of the input data value using a multiplier circuit as the product of the exponent of the compensated integer part and the approximate exponent of the compensated fractional part.

[0005] The embodiments disclosed herein provide a system for operating a computing device. The system includes: a separator circuit for separating an input data value into an integer part and a fractional part; a compensation circuit for generating a compensated fractional part based on at least one first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold; a scheduler circuit for dynamically determining the number of terms used to approximate the fractional part; and a Taylor expansion calculation circuit for calculating the sum of the Taylor expansion terms of the compensated fractional part. Attached Figure Description

[0006] The various aspects of this disclosure can be best understood in conjunction with the accompanying drawings and the following detailed description. Note that, in accordance with industry standard practice, the features are not drawn to scale. In fact, for clarity of discussion, the dimensions of the features may be arbitrarily increased or decreased.

[0007] Figure 1 This document describes an example of a neural network model relating to computational steps for performing a classification task, based on one or more embodiments described herein.

[0008] Figure 2 is a simplified diagram illustrating an exemplary structure of an exponent calculation circuit according to one or more embodiments described herein;

[0009] Figure 3 A simplified diagram illustrating an alternative exemplary structure of an exponent calculation circuit according to one or more embodiments described herein;

[0010] Figure 4 To illustrate one or more embodiments described herein Figure 3 A simplified diagram illustrating the exemplary structure of the separator circuit in the diagram;

[0011] Figure 5 To illustrate one or more embodiments described herein Figure 3 A simplified diagram illustrating the illustrative structure of the compensation circuit in the diagram;

[0012] Figure 6 To illustrate one or more embodiments described herein Figure 5 A simplified logic flowchart of the compensation circuit's workflow;

[0013] Figure 7 To illustrate the operation according to the embodiments described herein Figures 1 to 6 An illustrative logic flowchart of the exponent calculation circuit in the diagram;

[0014] Figure 8 To illustrate an embodiment according to this document, in the case of including Figures 1 to 7 A simplified diagram of a computing device that implements neural networks on an AI accelerator with a circuit structure.

[0015] Figure 9A and Figure 9B To illustrate one embodiment described herein Figures 1 to 8 The exponential calculation circuit in the figure is shown in the illustrative performance diagram of error and power efficiency.

[0016] [Symbol Explanation]

[0017] 100: Neural Network Model

[0018] 102: Input Data

[0019] 105: Neuron

[0020] 110: Floor

[0021] 115: Probability

[0022] 120: softmax operation

[0023] 130: Output

[0024] 202: Input data value

[0025] 202a: Symbol

[0026] 202b: Index

[0027] 202c: last digit

[0028] 204: Separator Circuit

[0029] 204a: Integer part

[0030] 204b: Decimal part

[0031] 210: LUT circuit

[0032] 212: Taylor term calculation circuit

[0033] 215: Index

[0034] 216: Index

[0035] 220: Multiplier

[0036] 230: Index value

[0037] 310: Compensation Circuit

[0038] 312: Compensation for the integer part

[0039] 313: Compensation for the decimal part

[0040] 314: Part

[0041] 315: Taylor term scheduler circuit

[0042] 316: Compensation for the integer part

[0043] 317: Compensation for the decimal part

[0044] 318: Taylor terms

[0045] 330: Output

[0046] 402: Shift Counter

[0047] 403: Number of shifted bits

[0048] 404: Shifter

[0049] 405: Decimal part

[0050] 407: Unsigned integer part

[0051] 408: Unsigned decimal part

[0052] 410: Symbol Combiner

[0053] 412: Standardized Circuits

[0054] 505: Absolute value circuit

[0055] 506, 508: Comparators

[0056] 509: Control Signal

[0057] 510: First Multiplexer

[0058] 512: Second Multiplexer

[0059] 600: Workflow

[0060] 602, 604, 606, 608, 610, 612: Steps

[0061] 700: Method

[0062] 702, 704, 706, 708, 710, 712: Steps

[0063] 800: Computing device

[0064] 810: Processor

[0065] 815: Data Interface

[0066] 820: Memory

[0067] 830: AI Accelerator

[0068] 831: Neural Networks

[0069] 840: Input

[0070] 850: Output Detailed Implementation

[0071] The following disclosure provides many different embodiments or examples for implementing various features of the invention. Specific examples of components and arrangements described below are used to simplify this disclosure. Of course, these are merely examples and are not intended to be limiting. For example, the following description of forming a first feature above or on a second feature may include embodiments in which the first and second features are in direct contact, and may also include embodiments in which an additional feature is formed between the first and second features such that the first and second features are not in direct contact. Furthermore, element symbols or letters may be repeated in various examples of this disclosure. This repetition is for simplicity and clarity and does not in itself specify a relationship between the various embodiments or configurations discussed.

[0072] Furthermore, for ease of description, spatial relative terms such as “below,” “under,” “below,” “above,” and “above” may be used herein to describe the relationship between one element or feature and another, as shown in the figures. In addition to the orientations shown in the figures, the spatial relative terms are intended to cover different orientations of the device during use or operation. The device may be oriented in other ways (rotated 90 degrees or otherwise), and the spatial relative descriptive terms used herein may be interpreted accordingly.

[0073] As used herein, the term “network” can include any framework based on hardware or software that contains any artificial intelligence network or system, neural network or system, and / or any training or learning model implemented thereon or together with it.

[0074] As used herein, the term "module" may encompass a hardware- or software-based framework that performs one or more functions. In some embodiments, a module may be implemented on one or more neural networks.

[0075] In recent years, the rapid development of artificial intelligence (AI) and machine learning has had a significant impact on various industries, from healthcare and finance to automotive and consumer electronics. As AI systems become increasingly complex, their computational demands also increase, driving the need for more efficient and powerful processing solutions. Traditional central processing units (CPUs) often struggle to keep up with these demands, leading to the widespread adoption of AI accelerators such as graphics processing units (GPUs) and tensor processing units (TPUs). GPUs and TPUs are better suited for AI applications than CPUs due to their ability to handle the massively parallel processing required for AI and machine learning tasks. Unlike CPUs, which are optimized for general-purpose computing, GPUs and TPUs are designed to execute thousands of operations simultaneously, making them ideal for handling large datasets and complex algorithms. This parallelism significantly accelerates the training and inference processes in AI models, resulting in faster and more efficient computation. Furthermore, GPUs and TPUs are optimized for the specific mathematical operations that support AI workloads, further enhancing performance in these applications.

[0076] AI accelerators have become a key component for deploying AI models, especially in tasks requiring massive parallel processing power, such as deep learning. These dedicated hardware components are designed to optimize the performance of AI workloads, resulting in faster processing times and more efficient resource utilization. However, this performance improvement often comes at the cost of higher power consumption, posing significant challenges to energy efficiency and thermal management.

[0077] This application relates to computing circuits, and more particularly to methods and apparatus for special application circuits used to calculate the exponent of an input data value. The embodiments described herein provide arithmetic logic unit (ALU) circuits for calculating the exponent of an input data value, such as 16-bit Brain Floating Point (BF16), 16-bit Half-Point Floating Point (FP16), and 6-bit floating point data types primarily used for machine learning and AI computation. In one embodiment, to calculate the exponent of an input data value, a separator circuit separates the input data value into an integer part and a fractional part. A compensation circuit is used to limit the fractional part to less than 0.5, thereby generating a compensated integer part and a compensated fractional part. A scheduler circuit, for example, dynamically determines the number of terms used to approximate the exponent of the fractional part based on the fractional part of the input data value, rather than using a predetermined number of Taylor expansion terms. A Taylor expansion calculation circuit calculates the sum of the number of Taylor expansion terms for the compensated fractional part. Then, the exponent of the input data value is calculated as the product of the exponent of the integer part and the approximate exponent of the fractional part.

[0078] In this way, exponential computing ALUs can be applied in AI accelerators as on-chip ALUs for complex operations based on exponential computation, such as softmax and SiLUs. This hardware-based computing allows for fast convergence and high-accuracy computation, as well as efficient circuit area utilization and low power consumption. Furthermore, compared to software-based computing on GPUs, hardware-based exponential computing ALU units require less GPU memory access.

[0079] Figure 1 Examples of neural network models 100 relating to computational steps for performing classification tasks according to one or more embodiments described herein are illustrated. In one embodiment, neural network 100 comprises a computational system built upon a collection of connection units or nodes (referred to as neurons 105). Neurons are typically connected by edges, and adjustable weights are typically associated with the edges. Neurons are typically aggregated into layers 110 such that different layers can perform different transformations on corresponding inputs and output transformed input data to the next layer.

[0080] For example, when each neuron receives an input signal, the input layer receives input data 102, performs a weighted sum on the input according to the weights assigned to each connection, and then applies the activation function associated with the corresponding neuron to the result. The output of the activation function is passed to the next layer of neurons or used as the final output of the network. The activation function can be the same or different between different layers. Indicative activation functions include, but are not limited to, Sigmoid, hyperbolic tangent, Rectified Linear Unit (ReLU), Leaky ReLU, Softmax, SiLU, etc. In this way, after several layers, the input data 102 received at the input layer is transformed into different values, indicating data features corresponding to the task that the neural network structure is designed to perform.

[0081] For example, input data 102 may contain images, and neural network 100 may be a classification model trained to classify objects in the input images. The output layer may output a probability 115, indicating the likelihood that the input image may contain one of a predefined object category, such as apple, orange, ..., dog, cat. A softmax operation 120 may be performed based on the output probability 115 to produce a final binary output 130 for classification. In this process, the operation of neural network 100 involves numerous exponential calculations, such as in the softmax operation, the SiLU operation, and / or similar operations.

[0082] Figure 2 is a simplified diagram illustrating an exemplary structure of a conventional exponent calculation circuit according to one or more embodiments described herein. For example, the exponent calculation circuit includes a separator circuit 204, a lookup table (LUT) circuit 210, a Taylor term calculation circuit 212, and a multiplier 220.

[0083] Given input data value 202, separator circuit 204 is used to separate the input data into an integer part 204a and a fractional part 204b. An illustrative circuit structure of separator circuit 204 is described below. Figure 4 Further details are provided below.

[0084] For the integer part 204a, the LUT circuit 210 retrieves the pre-stored exponent 215 of the integer 204a. For the fractional part 204b, the Taylor term calculation circuit 212 calculates the sum of a finite number of Taylor expansion terms of the fractional part 204b as an approximation of the exponent 216. Then, the multiplier 220 multiplies by the exponent 215 of the integer part and the exponent 216 of the fractional part to output the final exponent value 230.

[0085] In the exponent calculation circuit shown in Figure 2, the Taylor term calculation circuit 212 uses a predefined fixed number of Taylor expansion terms, such as N = 3, 4, 5, etc. Given a fixed number N, but with different fractional parts for different input data values ​​202, the accuracy of the exponent 216 may be sacrificed for exponents with larger fractional parts. On the other hand, using the same number N for smaller fractional parts wastes computational energy / cycles. Therefore, designing a Taylor term calculation circuit 212 with an optimal number of Taylor terms to balance computational cost / energy and accuracy remains challenging.

[0086] Figure 3 This is a simplified diagram illustrating an alternative exemplary structure of an exponent calculation circuit according to one or more embodiments described herein. The compensation circuit 310 is used to generate a compensated integer part (integer_c) 312 and a compensated fractional part (fractional_c) 313, instead of using a fixed number of Taylor expansion terms on the fractional part of the output of the separator circuit 204, such that 0 ≤ |fractional_c| ≤ 0.5. Additional structures and operations of the compensation circuit 310 can be described below. Figure 5 and Figure 6 Describe it.

[0087] In one embodiment, the Taylor term scheduler circuit 315 may receive all or at least a portion 314 of the compensated fractional part 313, and dynamically determine the number of Taylor terms required for that particular compensated fractional part 313. This dynamically determined number of Taylor terms 318 is then passed to the Taylor calculation circuit 212, which calculates only the dynamically determined N terms of the Taylor expansion of the exponent of the compensated fractional part 313. Tables 1 and 2 below provide examples of the dynamically determined N terms of the Taylor expansion of different fractional parts, for example, with and without compensation.

[0088] Table 1 Dynamic Taylor Term Settings (No Compensation)

[0089]

[0090] Table 2 Dynamic Taylor Term Settings (with Compensation)

[0091]

[0092] As shown in Tables 1 and 2, the compensation circuit 310 is used to limit the fractional part to less than 0.5, further reducing the number of Taylor expansion terms while maintaining calculation accuracy. Therefore, the combination of the compensation circuit 310 and the Taylor term scheduler circuit 315 together improves the calculation efficiency of the exponent calculation circuit.

[0093] For the compensated integer portion 312, the LUT circuit 210 retrieves the exponent of the compensated integer portion 316 in a manner similar to that described in FIG2. Then, the exponent of the integer portion 316 is multiplied by the exponent of the compensated fractional portion 317 by the multiplier 220 to produce an output 330 as the exponent of the input data value 202.

[0094] Figure 4 To illustrate one or more embodiments described herein Figure 3 A simplified diagram of an exemplary structure of the separator circuit 204 described herein. The separator circuit 204 may include a shift counter 402, a shifter 404, a sign combiner 410, and a normalization circuit 412.

[0095] In one embodiment, the input data value 202 (e.g., BF16 or FP16 data format) can be decomposed into a sign 202a, a mantissa 202c, and an exponent 202b. For example, when the input data value 202 is in BF16 format, bits 0 to 6 (7 bits) represent the mantissa 202c, bits 7 to 14 (8 bits) represent the exponent 202b, and the last bit represents the sign 202a. Then, the shift counter 402 can shift the exponent bits 202b by a certain number of bits to obtain the number of shifted bits 403 and the fractional flag portion 405 (indicating whether a fractional part exists). For example, if the magnitude of the input data value is less than 1, then the fractional flag 405frac_flag = 1, the unsigned integer portion 407 is set to 0, and the unsigned fractional portion 408 is equal to the input data value.

[0096] Then, the two outputs of shift counter 402, along with the mantissa bit 202c, are passed to shifter circuit 404. Shifter circuit 404 then shifts the bits to produce an unsigned integer part 407 and an unsigned fractional part 408.

[0097] The sign combiner circuit 410 can combine the sign 202a, the decimal mark 405, and the unsigned integer 407 to output the integer part 204a. The normalization circuit 412 can sequentially combine the sign 202a, the decimal mark 405, and the unsigned decimal part 408, and sequentially normalize the unsigned decimal part 408 to output the decimal part 204b.

[0098] Figure 5 To explain Figure 3 A simplified diagram of the exemplary structure of the compensation circuit 310 described herein, and Figure 6 Provided description of one or more embodiments according to the present document Figure 5 A simplified logic flowchart of the operation 600 of the compensation circuit 310 described herein is provided. For example, the compensation circuit 310 may include multiple comparators 506, 508 and multiple multiplexers 510, 512.

[0099] After the separator circuit separates the input data value 202 into an integer part 204a and a fractional part 204b, for example in Figure 6 In step 602, the magnitude (absolute value) of the fractional part 204b is obtained at the absolute value circuit 505. Then, comparator 506 compares the magnitude (absolute value) of the fractional part 204b with a predefined threshold of 0.5, for example, in... Figure 6 Step 604 in the process. Another comparator 508 compares the input data value 202 with 0, for example in... Figure 6 Step 606. The output of comparator 506 (e.g., indicating whether the magnitude (absolute value) of the fractional part 204b is greater than the first bit of 0.5) and the output of comparator 508 (e.g., indicating whether the second bit of the input data value 202 is greater than 0) can be concatenated into a two-bit control signal 509.

[0100] Control signal 509 is sent to the first multiplexer 510 and the second multiplexer 512 to select the output accordingly. For example, when control signal 509 is "00" or "01", indicating that |decimal| < 0.5, the first multiplexer 510 selects the integer part 204a as the output, and the second multiplexer 512 selects the decimal part 204b as the output, for example in... Figure 6 Step 608. In this case, the compensated integer part 312 and the compensated decimal part 313 are the same as the uncompensated integer part 204a and the uncompensated decimal part 204b, respectively.

[0101] For example, when control signal 509 is "10", it means |decimal| > 0.5, but the input data x < 0. Therefore, the first multiplexer 510 selects to output the integer part 204a minus 1, and the second multiplexer 512 selects to output the decimal part 204b plus 1. Figure 6 Step 612. At this time, the compensation integer part 312 and the compensation decimal part 313 are set to (uncompensated integer part 204a minus 1) and (uncompensated decimal part 204b plus 1), respectively.

[0102] For example, when control signal 509 is "11", indicating |decimal| > 0.5, but input data x > 0, the first multiplexer 510 selects to output the integer part 204a plus 1, and the second multiplexer 512 selects to output the decimal part 204b minus 1. Figure 6 Step 610. In this case, the compensation integer part 312 and the compensation decimal part 313 are set to (uncompensated integer part 204a plus 1) and (uncompensated decimal part 204b minus 1), respectively.

[0103] In this way, the fractional part 313 is limited to less than 0.5, which reduces the number of terms required for Taylor expansion calculations, thereby reducing the calculation cycle and energy consumption.

[0104] Figure 7 To illustrate the operation according to the embodiments described herein Figures 1 to 6 An exemplary logic flowchart of process 700 of the exponent calculation circuit described herein. One or more processes of method 700 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which, when run by one or more processors, may cause one or more processors to execute one or more processes. In some embodiments, method 700 corresponds to Figures 1 to 6 The operation of the exponent calculation circuit shown.

[0105] As shown in the figure, method 700 includes several enumeration steps, but the state of method 700 may include additional steps before, after, and between the enumeration steps. In some states, one or more enumeration steps may be omitted or executed in a different order.

[0106] In step 702, the separator circuit (e.g., Figure 3 The separator circuit 204 in the middle can convert the input data value (e.g., Figure 3 The input data value 202) is separated into integer parts (e.g., Figure 3 The integer part 204a) and the fractional part (e.g., Figure 3 (204b) The fractional part of the input data value. For example, the input data value includes: a first bit indicating the sign of the input data value; a first set of bits indicating the exponent of the input data value; and a second set of bits indicating the mantissa of the input data value.

[0107] In step 704, the compensation circuit (e.g., Figure 3 and Figure 5 The compensation circuit 310 in the circuit can be based on the first comparator circuit (e.g., Figure 5 The comparator 506 in the middle) at least the first output is used to generate the compensated fractional part (e.g., Figure 3 and Figure 5 The first comparator circuit will compensate the fractional part (313) in the first comparator circuit (e.g., the fractional part). Figure 5 The fractional part 204b in the first comparator circuit is compared with the first threshold. For example, the first comparator circuit (e.g., Figure 5 The comparator 506 in the circuit generates a first output by comparing the absolute value of the fractional part with a first threshold. The second comparator circuit (e.g., ...) Figure 5 The comparator 508 in the first multiplexer produces a second output by comparing at least the first bit of the input data value with zero. Figure 5The first multiplexer 510 in the middle can be based on the splicing of the first output and the second output (e.g., Figure 5 The control signal 509 in the middle selectively outputs a compensation integer part from the integer part, which is incremented or decremented by one. The second multiplexer (e.g., Figure 5 The second multiplexer 512 can selectively output a compensation fractional part from the fractional part based on the concatenation of the first output and the second output, which adds or subtracts one from the fractional part.

[0108] In step 706, the scheduler circuit (e.g., Figure 3 The Taylor term scheduler circuit 315 in the diagram dynamically determines the number of terms used to approximate the fractional part of the exponent. For example, the scheduler circuit may include one or more comparator circuits. Figure 3 (Not shown in the table) to compare the compensated decimal part with a corresponding predefined threshold, thereby generating a corresponding comparison output indicating whether the compensated decimal part is within the corresponding range. The scheduler circuit may further include a multiplexer to select one of a plurality of predefined quantities (e.g., 2, 4, 5, 6, as shown in Tables 1 and 2) based on the corresponding comparison outputs from one or more comparator circuits.

[0109] In step 708, the Taylor expansion calculation circuit (e.g., Figure 3 The Taylor calculation circuit 212 in the middle can calculate the sum of the Taylor expansion terms of the compensated fractional part as the approximate exponent of the compensated fractional part.

[0110] In step 710, the LUT circuit (e.g., Figure 3 The LUT circuit 210 in the memory cell can retrieve the exponent of the compensated integer part from the lookup table of the memory cell.

[0111] In step 712, the multiplier circuit (e.g., Figure 3 The multiplier 220 in the middle can calculate the exponent of the input data value as a compensation for the integer part (e.g., Figure 3 The exponent of the integer part 316) and the compensation fractional part (e.g., Figure 3 The product of the approximate exponents of the compensated decimal part (317) in the middle.

[0112] Figure 8 To illustrate an embodiment according to this document, in the case of including Figures 1 to 7 A simplified diagram of a computing device implementing neural networks on an AI accelerator, as described in the circuit structure. Figure 8As shown, computing device 800 includes a processor 810 coupled to memory 820. The operation of computing device 800 is controlled by processor 810. Furthermore, although only one processor 810 is shown in computing device 800, it should be understood that processor 810 may represent one or more central processing units, microcontrollers, multi-core processors, microprocessors, digital signal processors, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), graphics processing units (GPUs), and / or the like in computing device 800. Computing device 800 may be implemented as a standalone subsystem, a board added to a computing device, and / or a virtual machine.

[0113] Memory 820 may be used to store software executed by computing device 800 and / or one or more data structures used during operation of computing device 800. Memory 820 may contain one or more types of machine-readable media. Some common forms of machine-readable media may include floppy disks, floppy disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, any other optical media, punched cards, paper tape, any other physical media with a perforated pattern, RAM, PROM, EPROM, FLASH-EPROM, any other memory chips or cassette tapes, and / or any other media suitable for reading by a processor or computer.

[0114] The processor 810 and / or memory 820 can be configured in any suitable physical arrangement. In some embodiments, the processor 810 and / or memory 820 may be implemented on the same board, in the same package (e.g., a system-on-a-chip), on the same chip (e.g., a system-on-a-chip), etc. In some embodiments, the processor 810 and / or memory 820 may include distributed, virtualized, and / or containerized computing resources. Consistent with these embodiments, the processor 810 and / or memory 820 may reside in one or more data centers and / or cloud computing facilities.

[0115] In another embodiment, processor 810 may include multiple microprocessors, and / or memory 820 may include multiple registers and / or other memory elements, such that processor 810 and / or memory 820 can be configured in the form of a hardware-based neural network, such as... Figures 1 to 6 As further described in the text.

[0116] In some instances, memory 820 may contain non-transitory tangible machine-readable medium containing executable code that, when run by one or more processors (e.g., processor 810), causes one or more processors to perform methods described further in detail herein. For example, as shown, memory 820 contains instructions for operating neural network 831.

[0117] Memory 802 may be further coupled to AI accelerator 830, which may include an ALU, such as softmax, ReLU, SiLU, etc. The ALU of AI accelerator 803 may include one or more exponential calculation circuits, such as... Figures 3 to 6 As shown.

[0118] Data interface 815 may include a communication interface and a user interface (such as a voice input interface, a graphical user interface, etc.). For example, computing device 800 may receive input 840 (such as a training dataset) from a networked database via the communication interface. Alternatively, computing device 800 may receive input 840 (such as an input image) from a user via the user interface and generate output 850 (such as... Figure 1 (130 in the middle).

[0119] Some examples of computing devices, such as computing device 1400, may include non-transitory tangible machine-readable media containing executable code that, when run by one or more processors (e.g., processor 1410), causes one or more processors to perform the process of the method. Some common forms of machine-readable media that may contain the process of the method are, for example, floppy disks, floppy disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, any other optical media, punched cards, paper tape, any other physical media with a perforated pattern, RAM, PROM, EPROM, FLASH-EPROM, any other memory chips or cassette tapes, and / or any other media suitable for reading by a processor or computer.

[0120] The computing device 800 may be included in a system for running one or more neural networks. The system includes: a separator circuit for separating an input data value into an integer part and a fractional part; a compensation circuit for generating a compensated fractional part based on at least a first output of a first comparison circuit that compares the fractional part with a first threshold; a scheduler circuit for dynamically determining the number of terms for approximating the fractional part of the exponent; and a Taylor expansion calculation circuit for calculating the sum of the Taylor expansion terms of the compensated fractional part.

[0121] Figure 9A and Figure 9B To illustrate one embodiment described herein Figures 1 to 8The exponential calculation circuit described in the figure has an illustrative performance diagram of error and power efficiency. Figure 9A exhibit Figure 3 The computational circuit shown uses a compensation circuit 310 (e.g., "new dyn" in the illustration, with the standard deviation of the test random data "sign" = 1, 2, 4) to reduce computational energy by at least 18% compared to an uncompensated scheme (e.g., "dyn" in the illustration, with the standard deviation of the test random data "sign" = 1, 2, 4). Figure 9B exhibit Figure 3 The computational circuit shown, employing compensation circuit 310 (e.g., "new dyn" in the illustration, with the standard deviation of the test random data "sign" = 1, 2, 4), reduces error by 5.6 times and energy consumption by 30% compared to an uncompensated scheme (e.g., "dyn" in the illustration, with the standard deviation of the test random data "sign" = 1, 2, 4).

[0122] In one exemplary embodiment, this disclosure relates to a computational apparatus for performing input data computation in a neural network. The apparatus includes: a separator circuit for separating an input data value into an integer part and a fractional part; a compensation circuit for generating a compensated fractional part based on at least a first output of a first comparator circuit that compares the fractional part with a first threshold; a scheduler circuit for dynamically determining the number of terms for approximating the fractional part of the exponent; and a Taylor expansion calculation circuit for calculating the sum of the Taylor expansion terms of the compensated fractional part.

[0123] In some embodiments, the input data value includes: a first bit indicating the sign of the input data value; a first set of bits indicating the exponent of the input data value; and a second set of bits indicating the mantissa of the input data value.

[0124] In some embodiments, the separator circuit includes: one or more shifter circuits for generating an unsigned integer portion and an unsigned fractional portion from a first set of bits and a second set of bits; a sign combiner circuit for generating an integer portion by combining the first bit with the unsigned integer portion; and a normalization circuit for generating a fractional portion based on the first bit and the unsigned fractional portion.

[0125] In some embodiments, the compensation circuit includes: a first comparator circuit that generates a first output by comparing the absolute value of the fractional part with a first threshold; a second comparator circuit that generates a second output by comparing at least one bit of the input data value with zero; a first multiplexer that selectively outputs a compensated integer part from an integer part, an integer part plus one, or a fractional part minus one, based on the concatenation of the first output and the second output; and a second multiplexer that selectively outputs a compensated fractional part from a fractional part, a fractional part plus one, or a fractional part minus one, based on the concatenation of the first output and the second output.

[0126] In some embodiments, the scheduler circuit includes: one or more comparator circuits, each comparator circuit comparing a compensated decimal portion with a corresponding predefined threshold to generate a corresponding comparison output indicating whether the compensated decimal portion is within a corresponding range; and a multiplexer that selectively outputs one of a plurality of predefined quantities based on the corresponding comparison outputs from the one or more comparator circuits.

[0127] In some embodiments, the Taylor expansion calculation circuit calculates the sum of the Taylor expansion terms as an approximate exponent that compensates for the fractional part.

[0128] In some embodiments, the circuit further includes a microcontroller that retrieves an exponent for the compensated integer portion from a lookup table of memory cells.

[0129] In some embodiments, the circuit further includes a multiplier circuit that outputs the exponent of the input data value as the product of the exponent of the compensated integer part and the approximate exponent of the compensated decimal part.

[0130] In some embodiments, the circuit further includes an artificial intelligence (AI) accelerator circuit, which includes an arithmetic logic unit (ALU) that calculates a transformation of the input data value based on an exponent of the input data value when the input data value is part of the neural network input.

[0131] In another exemplary embodiment, one embodiment of this disclosure relates to a method of operating an application-specific integrated circuit (ASIC) to perform input data computation. The method includes: separating an input data value into an integer part and a fractional part via a separator circuit; generating a compensated fractional part via a compensation circuit based on at least a first output of a first comparison circuit that compares the fractional part with a first threshold; dynamically determining the number of terms for approximating the fractional part using a scheduler circuit; calculating the sum of the Taylor expansion terms of the compensated fractional part as an approximate exponent of the compensated fractional part via a Taylor expansion calculation circuit; and outputting the exponent of the input data value via a multiplier circuit as the product of the exponent of the compensated integer part and the approximate exponent of the compensated fractional part.

[0132] In some embodiments, the input data value includes: a first bit indicating the sign of the input data value; a first set of bits indicating the exponent of the input data value; and a second set of bits indicating the mantissa of the input data value.

[0133] In some embodiments, the step of generating a compensated decimal part via a compensation circuit includes the following steps: generating a first output by comparing the absolute value of the decimal part with a first threshold via a first comparator circuit; generating a second output by comparing at least one bit of the input data value with zero via a second comparator circuit; selectively outputting a compensated integer part from an integer part, an integer part plus one, or a decimal part minus one via a first multiplexer based on the concatenation of the first output and the second output; and selectively outputting a compensated decimal part from a fractional part, a fractional part plus one, or a decimal part minus one via a second multiplexer based on the concatenation of the first output and the second output.

[0134] In some embodiments, the step of dynamically determining the number of items by the scheduler circuit includes the following steps: comparing the compensated decimal portion with a corresponding predefined threshold by each of one or more comparator circuits to generate a corresponding comparison output indicating whether the compensated decimal portion is within a corresponding range; and selectively outputting one of a plurality of predefined quantities by a multiplexer based on the corresponding comparison outputs from one or more comparator circuits.

[0135] In some embodiments, the method of operation further includes the step of retrieving an exponent of the compensated integer portion from a lookup table of memory cells.

[0136] In some embodiments, the operation method further includes the step of: outputting the exponent of the input data value through the multiplier circuit as the product of the exponent of the compensated integer part and the approximate exponent of the compensated decimal part.

[0137] In another exemplary embodiment, an embodiment of this disclosure relates to a system operating one or more neural networks. The system includes: a separator circuit for separating an input data value into an integer part and a fractional part; a compensation circuit for generating a compensated fractional part based on at least a first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold; a scheduler circuit for dynamically determining the number of terms for approximating the fractional part of the exponent; and a Taylor expansion calculation circuit for calculating the sum of the Taylor expansion terms of the compensated fractional part.

[0138] In some embodiments, the compensation circuit includes: a first comparator circuit that generates a first output by comparing the absolute value of the fractional part with a first threshold; a second comparator circuit that generates a second output by comparing at least one bit of the input data value with zero; a first multiplexer that selectively outputs a compensated integer part from an integer part, an integer part plus one, or a fractional part minus one, based on the concatenation of the first output and the second output; and a second multiplexer that selectively outputs a compensated fractional part from a fractional part, a fractional part plus one, or a fractional part minus one, based on the concatenation of the first output and the second output.

[0139] In some embodiments, the scheduler circuit includes: one or more comparator circuits, each comparator circuit comparing a compensated decimal portion with a corresponding predefined threshold to generate a corresponding comparison output indicating whether the compensated decimal portion is within a corresponding range; and a multiplexer for selectively outputting one of a plurality of predefined quantities based on the corresponding comparison outputs from the one or more comparator circuits.

[0140] In some embodiments, the Taylor expansion calculation circuit calculates the sum of the Taylor expansion terms as an approximate exponent that compensates for the fractional part. The system further includes a microcontroller for retrieving the exponent that compensates for the integer part from a lookup table in the memory cell.

[0141] In some embodiments, the system further includes a multiplier circuit for outputting the exponent of the input data value as the product of the exponent of the compensated integer part and the approximate exponent of the compensated decimal part.

[0142] The foregoing outlines features of several embodiments to enable those skilled in the art to better understand the various aspects of this disclosure. Those skilled in the art should understand that this disclosure can be readily used as a basis for designing or modifying other processes and structures to achieve the same purposes and / or advantages as the embodiments described herein. Those skilled in the art should also recognize that these equivalent constructions do not depart from the spirit and scope of this disclosure, and that various changes, substitutions, and modifications can be made to these equivalent constructions without departing from the spirit and scope of this disclosure.

Claims

1. A computing device for performing calculations on input data in a neural network, characterized in that, It includes: A separator circuit for separating an input data value into an integer part and a fractional part; A compensation circuit is used to generate a compensated fractional part based on at least one first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold. A scheduler circuit is used to dynamically determine the number of terms used to approximate the fractional part of an exponent; as well as A Taylor expansion calculation circuit is used to calculate the sum of the Taylor expansion terms of the compensated fractional part.

2. The computing device as described in claim 1, characterized in that, The input data value includes: a first bit indicating a sign of the input data value; a first set of bits indicating an exponent of the input data value; and a second set of bits indicating a mantissa of the input data value.

3. The computing device as described in claim 2, characterized in that, The separator circuit includes: One or more shifter circuits for generating an unsigned integer part and an unsigned fractional part from the first set of bits and the second set of bits; A sign combiner circuit for generating the integer part by combining the first bit with the unsigned integer part; as well as A standardized circuit for generating the fractional part based on the first bit and the unsigned fractional part.

4. The computing device as described in claim 1, characterized in that, The compensation circuit includes: The first comparator circuit is used to generate the first output by comparing an absolute value of the fractional part with a first threshold. A second comparator circuit is used to generate a second output by comparing at least one bit of the input data value with zero; A first multiplexer is configured to selectively output a compensated integer part from the integer part, the integer part plus one, or the integer part minus one, based on a concatenation of the first output and the second output; as well as A second multiplexer is configured to selectively output the compensated fractional part from the fractional part, the fractional part plus one, or the fractional part minus one, based on the concatenation of the first output and the second output.

5. The computing device as claimed in claim 1, characterized in that, The scheduler circuit includes: One or more comparator circuits, each comparing the compensated decimal part with a corresponding predefined threshold to generate a corresponding comparison output indicating whether the compensated decimal part is within a corresponding range; and A multiplexer for selectively outputting one of a plurality of predefined quantities based on the corresponding comparison output from the one or more comparator circuits.

6. The computing device as claimed in claim 1, characterized in that, The Taylor expansion calculation circuit calculates the sum of the Taylor expansion terms as an approximate exponent of the compensated decimal part.

7. The computing device as claimed in claim 1, characterized in that, The computing device further includes: A microcontroller for retrieving an exponent with a compensated integer portion from a lookup table in a memory cell.

8. The computing device as claimed in claim 7, characterized in that, The computing device further includes: A multiplier circuit is used to output an exponent of the input data value as a product of the exponent of the compensated integer part and the approximate exponent of the compensated decimal part.

9. A method for operating a computing device, characterized in that, Include: A demultiplexer circuit separates an input data value into an integer part and a fractional part. A compensation fractional part is generated by a compensation circuit based on at least one first output of a first comparator circuit, which compares the fractional part with a first threshold. A scheduler circuit is used to dynamically determine the number of terms used to approximate the fractional part of an exponent. as well as A Taylor expansion calculation circuit is used to calculate the sum of the Taylor expansion terms of the compensated fractional part as an approximate exponent of the compensated fractional part; and A multiplier circuit outputs an exponent of the input data value as a product of an exponent of the compensated integer part and an approximate exponent of the compensated decimal part.

10. A system for operating a computing device, characterized in that, Include: A separator circuit for separating an input data value into an integer part and a fractional part; A compensation circuit is used to generate a compensated fractional part based on at least one first output of a first comparator circuit, the first comparator circuit comparing the fractional part with a first threshold. A scheduler circuit is used to dynamically determine the number of terms used to approximate the fractional part of an exponent; as well as A Taylor expansion calculation circuit is used to calculate the sum of the Taylor expansion terms of the compensated fractional part.

Images