Floating-point calculation system, method, and program with threshold prediction

Abstract

The system includes a floating-point computation unit configured to perform a dot product operation according to a first floating-point value and a second floating-point value, and detection logic operably coupled to the floating-point computation unit. The detection logic is configured to calculate a difference between a fixed-point sum of the exponents of the first floating-point value and the second floating-point value, and to detect the existence of a condition prior to completion of the dot product operation by the floating-point computation unit based on the calculated difference. In response to detecting the existence of the condition, the detection logic is further configured to cause the floating-point computation unit to avoid performing a subset of calculations that would normally be performed as part of the dot product operation. Such a technique serves as a predictor that the avoided subset of calculations would result in a dot product of two floating-point values ​​being below a threshold.

Description

[Technical field]

[0001] The present invention relates generally to performing floating-point calculations for use in artificial intelligence systems. [Background technology]

[0002] Machine learning algorithms, considered as an example of artificial intelligence systems, are becoming widely used in computational applications that benefit from improved iterative computational accuracy. Neural network-based algorithms are one of the most widely used types of machine learning algorithms. A neural network is a model that recognizes underlying relationships in a set of data through a process that mimics the way the human brain operates. A neural network model is first trained using a training data set (training phase), and then the trained neural network model is used to recognize relationships in a target data set (inference phase). While the inference phase can rely on low-precision fixed-point arithmetic, the training phase typically requires floating-point arithmetic. Summary of the Invention

[0003] In one illustrative embodiment, a system comprises a floating-point computation unit configured to perform a dot product operation according to a first floating-point value and a second floating-point value, and detection logic operably coupled to the floating-point computation unit. The detection logic is configured to calculate a difference between a fixed-point sum of exponents of the first floating-point value and the second floating-point value, and detect the presence of a condition prior to completion of the dot product operation by the floating-point computation unit based on the calculated difference. In response to detecting the presence of the condition, the detection logic is further configured to cause the floating-point computation unit to avoid performing a subset of calculations that would otherwise be performed as part of the dot product operation.

[0004] Further illustrative embodiments are provided in the form of an apparatus including a processor and memory configured to execute instruction code, a method configured to perform the detection and avoidance steps, and a non-transitory processor-readable storage medium embodied with executable instruction code that, when executed by a processor, causes the processor to perform the steps.

[0005] Advantageously, as one example, the illustrative embodiments predict negative dot product outputs or acceptably small positive dot product outputs before the dot product is calculated in order to save computational overhead early on in terms of the hardware used to support the computational algorithm.

[0006] These and other features and advantages of the embodiments described herein will become more apparent from the accompanying drawings and the detailed description that follows. [Brief description of the drawings]

[0007] [Figure 1A] FIG. 1 illustrates a floating-point field format by which one or more illustrative embodiments may be implemented. [Figure 1B] 1 illustrates an arithmetic representation of floating-point numbers, with which one or more illustrative embodiments may be implemented. [Figure 1C] FIG. 1 illustrates a computation kernel associated with floating-point dot product computation, by which one or more illustrative embodiments may be implemented. [Diagram 2] FIG. 1 depicts floating-point multiply-accumulator logic by which one or more illustrative embodiments may be implemented. [Diagram 3] FIG. 1 illustrates threshold detection logic for floating-point dot product calculations, in accordance with an illustrative embodiment. [Figure 4] 1A-1B depict process flows without (A) and with (B) voltage scaling, in accordance with an illustrative embodiment. [Diagram 5] FIG. 1 illustrates a method for threshold detection logic for floating-point dot product calculations, in accordance with an illustrative embodiment. [Figure 6] FIG. 1 illustrates an exemplary implementation of an artificial intelligence system in accordance with an illustrative embodiment. [Figure 7] A diagram depicting an exemplary processor system in accordance with an illustrative embodiment. [Figure 8] FIG. 1 is a diagram illustrating a cloud computing environment in accordance with an illustrative embodiment. [Figure 9] FIG. 1 illustrates an abstraction model layer in accordance with an illustrative embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Illustrative embodiments may be described herein with respect to exemplary computing environments, cloud infrastructures, data repositories, data centers, data processing systems, information processing systems, computer systems, data storage systems, and associated servers, computers, storage units, and devices, as well as other processing and computing devices. However, it should be understood that embodiments of the invention are not limited to use with the particular illustrative system and device configurations shown. Furthermore, terms such as "cloud platform," "cloud computing environment," "cloud infrastructure," "data repository," "data center," "data processing system," "information processing system," "computer system," "data storage system," "computing environment," and the like, used herein, are intended to be broadly interpreted to encompass other types of systems, including, for example, private and / or public cloud computing or storage systems, as well as distributed virtual infrastructures. However, a given embodiment may more generally include any arrangement of one or more processing devices.

[0009] As discussed above in the Background section, the training phase of neural network models used in artificial intelligence (e.g., machine learning) systems typically requires floating-point arithmetic calculations. As an example, such floating-point arithmetic may include calculations defined in the Institute of Electrical and Electronics Engineers (IEEE) standard entitled "IEEE 754-2019: IEEE Standard for Floating-Point Arithmetic." The IEEE 754 standard specifies arithmetic formats and methods for binary (base 2) and decimal (base 10) floating-point arithmetic in a computer programming environment. The IEEE indicates that implementations of floating-point systems conforming to this standard may be realized entirely in software, entirely in hardware, or in any combination of software and hardware.

[0010] IEEE 754 defines a half-precision format using 16 bits per number, a single-precision format with 32 bits per number, and a double-precision format with 64 bits per number. Each format includes a sign "s", an exponent "e", and a mantissa expressed as a fractional value "f" (e.g., the number is normalized by assuming one leading bit followed by a fractional part that contains the significant value to the right of the decimal point). Thus, the mantissa is the part of a floating-point number that represents the significant digits of the number, and is multiplied by the base raised to the exponent to give the actual value of the number.

[0011] Figure 1A illustrates a floating-point number field format 100 that conforms to the IEEE 754 standard and in which one or more illustrative embodiments may be implemented. The variables n and p, which indicate the number of exponent and fraction bits, respectively, depend on the selected precision. Figure 1B illustrates an arithmetic representation 110 of a floating-point number, denoted as a value (x), that conforms to the field format 100 of Figure 1A and in which one or more illustrative embodiments may be implemented.

[0012] One of the floating-point calculations performed during the training phase of neural network models used in artificial intelligence systems is the floating-point dot product calculation. The dot product calculation is usually performed in the multiply-accumulator (MAC) unit of an artificial intelligence system. The dot product is an algebraic operation that takes two equal-length number sequences (e.g., two floating-point numbers) as input and returns a single number. More specifically, the dot product is the sum of the products of corresponding entries of the two sequences. If the two input numbers are vectors, the result of the dot product operation (also called the inner product) is a scalar value.

[0013] Furthermore, the computational kernel in modern artificial intelligence systems takes the inner product (the result of a floating-point dot product operation) and applies the Rectified Linear Unit (Relu) function to the inner product. Figure 1C shows the inner product y computed for two floating-point values ​​x and w.

number

[0014] In this specification, the dot product

number

[0015] Recall that, as described above and shown in Figures 1A and 1B, floating-point numbers are represented using three fields: sign (s), exponent (e), and fraction (f). For the IEEE754 half-precision format, for example, s=1 bit, e=5 bits, and f=10 bits. A floating-point number has a value (x)=(-1) for normalized numbers. s *2 [e] *(1.f). As is evident, the magnitude of the value depends primarily on the smaller number of bits, e, and the longer bit stream, f, acts to fine-tune the magnitude of the value. For example, the magnitude of a number x is always 2 [e] ≦|x|<2 [e+1] It is appreciated herein that this property advantageously, in accordance with illustrative embodiments, provides significant opportunities for zero anticipation in dot product computations.

[0016] The product of two floating-point numbers X and Y is restricted as follows: X=(-1) s *2 [ex] *(1.fx) => 2 [ex] ≦|X|<2 [ex+1] Y=(-1) s *2 [ey] *(1.fy) => 2 [ey] ≦|Y|<2 [ey+1] ∴2 [ex+ey] ≦|XY|<2 [ex+ey+2]

[0017] It is further understood herein that, according to an illustrative embodiment, the calculation of the maximum and minimum range of a product does not require multiplications, but only one fixed-point addition of exponent values ​​for two floating-point values ​​X and Y, namely ex+ey. In a dot product calculation, many (N) such products are summed, e.g.

number

[0018] The total ranges for the positive and negative groups are as follows: Sum of positive products:

number

number

[0019] Thus, in the illustrative embodiment, the sum of the products

number

number

[0020] Note that the required calculation in equation (1) above involves only fixed-point sums, without expensive multiplications or floating-point calculations. Advantageously, this computationally inexpensive pre-check avoids nearly 50 percent (%) of the calculations (when negative numbers occur). If the number is found to be positive, normal floating-point calculations continue.

[0021] As mentioned above, floating-point dot-product calculations are typically performed in a MAC hardware unit associated with an artificial intelligence system. FIG. 2 illustrates floating-point multiply-add logic associated with a MAC unit 200, by which one or more illustrative embodiments may be implemented. The MAC unit combines three values ​​A, B, and C by adding or subtracting value C to the dot product of values ​​A and B. For floating-point values, the MAC unit 200 processes inputs EA, EB, and EC, where E refers to the exponent portion of values ​​A, B, and C, respectively. The MAC unit 200 also processes inputs MA, MB, and MC, where M refers to the mantissa (fraction) portion of values ​​A, B, and C, respectively. The floating-point MAC unit 200 is relatively complex, including alignment logic, adder logic, multiplier logic, shifter logic, comparator logic, and other functional logic (as illustrated in FIG. 2). The conventional logical operation of a multiply-accumulate unit such as floating-point MAC unit 200 will be understood by those skilled in the art, and therefore will not be described in further detail herein.

[0022] Assuming that the fixed-point addition energy is 10% of the floating-point based MAC operation, it is understood herein that by predicting negative numbers with 10% prediction overhead, about 40% of the MAC calculations can be skipped. Meanwhile, if the accumulated number turns out not to meet the pre-check condition, the 10% prediction overhead is wasted. Hence, 0.4*(100% savings-10% overhead)+0.6*(-10% overhead)=30% energy savings.

[0023] It is understood herein that in some circumstances, equation (1) may be more conservative than necessary or desirable since the maximum possible magnitude of the positive product is compared to the minimum possible magnitude of the negative product. Therefore, in an alternative embodiment, the pre-check condition is relaxed by replacing the "0" in the right-hand term of equation (1) with a threshold value "Th", as depicted below in equation (2).

number

[0024] It should be understood that Th>0 and the magnitude of Th is generally set relatively small. In this way, the left term is a positive number, but if it is very close to 0, the calculation can be skipped. By controlling the Th value, more energy efficiency is achieved at the expense of acceptably lower accuracy. That is, if an additional 10% of the calculations can be avoided based on the setting of Th, the energy savings is 0.5*(100% savings-10% overhead)+0.5*(-10% overhead)=40% energy savings.

[0025] 3 depicts a logic implementation 300 of threshold detection for floating-point dot-product calculations, according to an illustrative embodiment. The logic implementation 300 represents one illustrative embodiment of a hardware implementation of equation (2) above. Alternative embodiments may have other logic implementations.

[0026] As shown, logic implementation 300 comprises a threshold detector 310 (detection logic) operably coupled to a floating point MAC unit 340, which is operably coupled to a multiplexer 342. Threshold detector 310 comprises logic components including adder 312, adder 314, demultiplexer 316, register 320, register 322, multiplexer 324, exclusive-or (XOR) gate 326, adder 328, and comparator 330. When a data input, data output, selection bit, control bit (or digital signal), etc. is referred to as being input or output from a given logic component, this assumes that the given logic component has corresponding terminals that enable connectivity with other logic components for receiving corresponding such inputs and / or sending such outputs.

[0027] As explained above, it is important to note that the calculation of the maximum and minimum range of the product of two floating point values ​​X and Y does not require any multiplications, but only one addition of the exponent values ​​for the two floating point values ​​X and Y, namely ex+ey, and that in a dot product calculation, many (N) such products are summed together, e.g.

number

[0028] The demultiplexer 316 selects a positive result on the first output.

number

number

[0029] As further shown in FIG. 3, an XOR gate 326 outputs sx, which are sign bits for the floating-point values ​​X and Y, respectively. i and sy ias input. When sx and sy are different, this means that the sign of the product of X and Y is negative; therefore, the value is accumulated with the value in the negative register 322. When sx and sy are the same, this means that the sign of the product of X and Y is positive; therefore, the value is accumulated with the value in the positive register 320. Note that the output of the XOR gate 326 is therefore used to control the demultiplexer 316 and the multiplexer 324, as appropriate.

[0030] The positive result from positive register 320 and the negative result from register 322 are also input to adder 328, whose output is checked against threshold Th in comparator 330. This is where the pre-check condition of equation (2) is determined. The output of comparator 330 is provided as an input to EN of floating-point MAC unit 340 and also as a control signal to multiplexer 342, which has a first input that is 0 and a second input that is the output of floating-point MAC unit 340. Note that EN represents an enable switch for floating-point MAC unit 340 such that floating-point MAC unit 340 operates only when the output of comparator 330 is logic 1 (EN=1). Thus, assuming floating-point MAC unit 340 is configured as shown in floating-point MAC unit 200 of Figure 2, the input values ​​(i.e., EA, EB, MA, MB...) will not be updated in the input registers when EN = 0. In this manner, the circuitry of unit 200 of Figure 2 will be inactive and maintain its previous computation status.

[0031] Multiplexer 342 represents the Relu function discussed above. Thus,

number

[0032] It should be appreciated that in additional embodiments, the circuitry in threshold detector 310 can be implemented with a voltage scaling function such that threshold detector 310 operates below a typical supply voltage to conserve energy. Such voltage scaling function is controlled by voltage scaling controller 344. Controller 344 is configured to enable or disable the voltage scaling function as needed or desired.

[0033] 4A and 4B depict a process flow 400 without voltage scaling and a process flow 410 with voltage scaling, respectively, in accordance with an illustrative embodiment. Process flows 400 and 410 correspond to the processing of sequential data (data 1, data 2, data 3, data 4, etc.) involving threshold detector 310 and floating point MAC unit 340. For example, when floating point MAC unit 340 performs a calculation on a given data (data 1), threshold detector 310 performs a prediction (based on threshold detection) for the next data (data 2), and so on.

[0034] Because the prediction stage calculations performed by the threshold detector 310 are relatively simple, the processing delay is relatively small compared to the processing delay associated with the floating point processing of the MAC unit, for example, this relative processing time difference is evident in FIG. 4A. Thus, voltage scaling can be applied to the prediction operation, for example, the voltage scaling controller 344 causes a percentage reduction in the logic supply voltage VDD (operating or reference voltage) supplied to the circuitry of the threshold detector 310. In some embodiments, the VDD is reduced by 30% of a typical (normal) voltage operating level. The reduced voltage causes the threshold detector 310 to operate proportionately slower, i.e., the processing delay increases. In this case, it is also possible that the time period (time delay) for the floating point calculations is fully utilized by the prediction stage, as depicted in the process flow 410 of FIG. 4B. For example, the processing time for the prediction for data 2 is slowed down to take up all or most of the processing time for the floating point calculations for data 1.

[0035] In some embodiments, by reducing the VDD for prediction operations, the overhead is reduced by half (energy=C*V DD 2 By the formula, 0.7*0.7=0.49), resulting in a 45% energy savings. Such energy savings can be significant since deep neural networks consist of multiple layers (e.g., up to 50), each of which includes convolutions, optional batch normalization, and Relu functions. Scaling of VDD can be applied by the controller 344 using any typical voltage control mechanism.

[0036] 5 illustrates a method 500 of threshold detection for floating-point dot-product calculations, according to an illustrative embodiment. It should be appreciated that method 500 may be performed, for example, by a system that includes logic implementation 300 of FIG. 3. However, in alternative embodiments, method 500 may also be performed by other systems configured to perform the steps of the method.

[0037] In step 502, a difference is calculated between the fixed-point sum of the exponents of the first floating-point value and the second floating-point value.

[0038] In step 504, prior to completion of the dot product operation by a floating-point computation unit configured to perform the dot product operation according to the first floating-point value and the second floating-point value, the existence of a condition is detected based on the calculated difference.

[0039] Step 506, in response to detecting the existence of the condition, causes the floating point computation unit to avoid performing a subset of the computations that would normally be performed as part of the dot product operation.

[0040] FIG. 6 illustrates an exemplary implementation of an artificial intelligence system 600, according to an illustrative embodiment. As shown, the system 600 includes a training data set 610, a neural network model 620, a threshold detection logic 630, a floating-point MAC unit 640, and a rectified linear unit 650. With respect to the exemplary hardware implementation illustrated in FIG. 3 and described above, it should be appreciated that the adder 312, the adder 314, the demultiplexer 316, the register 320, the register 322, the multiplexer 324, the XOR gate 326, the adder 328, the comparator 330, and the controller 344 can be implemented as part of the threshold detection logic 630. Furthermore, the floating-point MAC unit 340 can be implemented by the floating-point MAC unit 640. Furthermore, the multiplexer 342 can be implemented by the rectified linear unit 650. It should be appreciated that the threshold detection logic 630, the floating point MAC unit 640, and the rectified linear unit 650 are used to perform calculations during training of the neural network model 620 based on the training data set 610.

[0041] In an exemplary embodiment, the threshold detection logic 630, the floating point MAC unit 640, and the rectifier linear unit 650 of the artificial intelligence system 600 are implemented by one or more application specific integrated circuits (ASICs). An ASIC is an integrated circuit (IC) chip or device that is customized for a specific purpose, comprising logic (e.g., circuits, processors, memory, etc.) that is programmed or otherwise configured with executable program code (e.g., instruction code, computer program code, etc.) for a specific purpose. In this exemplary case, the specific purpose is the implementation and execution of an artificial intelligence system (e.g., a machine learning algorithm), and more specifically, the training phase of a neural network model 620 using a training data set 610. An ASIC may also be considered a system on a chip (SoC). Some ASIC implementations that may be used with one or more illustrative embodiments employ cell libraries of user-selectable basic logic functions (e.g., multiplexers and comparators configured with multiple VLSI transistor devices to provide various functions such as switching and comparison) to allow configuration (and reconfiguration) of the system.

[0042] It is further understood that artificial intelligence system 600, and portions thereof, may be realized in alternative circuit / processor-based technologies, such as technologies including one or more multi-core central processing units (CPUs), one or more graphics processing units (GPUs), and one or more field programmable gate arrays (FPGAs). In some embodiments, artificial intelligence system 600 may be implemented as a combination of two or more circuit / processor-based technologies (e.g., ASICs, CPUs, GPUs, FPGAs, etc.).

[0043] The techniques depicted in Figures 1-6 may also include providing a system, as described herein, including individual software modules, each of which is embodied in a tangible computer-readable recordable storage medium. All of the modules (or any subset thereof) may be on the same medium, or, for example, each module may be on a different medium. The modules may include any or all of the components shown in the figures and / or described herein. In one embodiment of the invention, the modules may run on, for example, a hardware processor. In this case, the method steps may be implemented using individual software modules of the system as described above running on the hardware processor. Additionally, a computer program product may include a tangible computer-readable recordable storage medium having code adapted to be executed to implement at least one method step described herein, including provisioning the individual software modules on the system.

[0044] Additionally, the techniques depicted in Figures 1-6 may be implemented via a computer program product that may include computer usable program code stored in a computer readable storage medium in a data processing system, the computer usable program code being downloaded over a network from a remote data processing system. Also, in one embodiment of the present invention, the computer program product may include computer usable program code stored in a computer readable storage medium in a server data processing system, the computer usable program code being downloaded over a network to the remote data processing system for use in the computer readable storage medium by the remote system.

[0045] An embodiment of the present invention or elements thereof may be implemented in the form of an apparatus comprising a memory and at least one processor coupled to the memory and configured to perform the exemplary method steps.

[0046] In addition, an embodiment of the present invention may utilize software running on a computer or workstation. With reference to FIG. 7, such an implementation may employ, for example, a processor 702, a memory 704, and an input / output interface formed, for example, by a display 706 and a keyboard 708. The term "processor" herein is intended to include any processing device, such as, for example, a multi-core CPU, a GPU, an FPGA, or other forms of processing circuitry, such as one or more ASICs, or combinations thereof. Additionally, the term "processor" may refer to more than one individual processor. The term "memory" is intended to include memory associated with a processor (e.g., a CPU, a GPU, an FPGA, an ASIC, etc.), such as, for example, a RAM (random access memory), a ROM (read only memory), a fixed memory device (e.g., a hard drive), a removable memory device (e.g., a diskette), a flash memory, etc. Additionally, the term "input / output interface" herein is intended to include, for example, a mechanism for inputting data into a processing unit (e.g., a mouse), and a mechanism for providing results related to the processing unit (e.g., a printer). The processor 702, memory 704, and input / output interfaces such as a display 706 and a keyboard 708 may be interconnected, for example, via a bus 710 as part of a data processing unit 712. Suitable interconnections, for example, via the bus 710, may also be provided to a network interface 714, such as a network card, that may be provided for interfacing with a computer network, and to a media interface 716, such as a diskette or CD-ROM drive, that may be provided for interfacing with media 718.

[0047] Thus, computer software containing instructions or code for carrying out the methods of the present invention as described herein may be stored in associated memory devices (e.g., ROM, fixed or removable memory) and may be loaded partially or entirely (e.g., into RAM) and implemented by a CPU when ready for use. Such software may include, but is not limited to, firmware, resident software, microcode, etc.

[0048] A data processing system suitable for storing and / or executing program code will comprise at least one processor 702 coupled directly or indirectly to memory elements 704 through a system bus 710. The memory elements may include local memory utilized during the actual implementation of the program code, bulk storage, and cache memory that provides temporary storage of at least some of the program code to reduce the number of times the code must be retrieved from bulk storage during implementation.

[0049] Input / output or I / O devices (including but not limited to a keyboard 708, a display 706, a pointing device, etc.) may be coupled to the system either directly (such as via a bus 710) or through intervening I / O controllers (omitted for clarity).

[0050] Network adapters, such as network interface 714, may be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters.

[0051] As used herein, including in the claims, a "server" includes a physical data processing system that executes a server program (e.g., system 712 shown in FIG. 7). It will be understood that such a physical server may or may not include a display and keyboard.

[0052] The present invention may be a system, method, or computer program product, or a combination thereof, at any possible level of integration of technical detail. The computer program product may include a computer-readable storage medium or media having computer-readable program instructions for causing a processor to implement aspects of the present invention.

[0053] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, a device on which the instructions are recorded and mechanically encoded such as punch cards or ridges in a groove structure, and any suitable combination thereof. Computer-readable storage media in this specification should not be interpreted as signals that are inherently ephemeral, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or electrical signals transmitted through wires.

[0054] The computer readable program instructions described herein may be downloaded from a computer readable storage medium into each computing / processing device, or may be downloaded to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers these computer readable program instructions to be stored in a computer readable storage medium within the respective computing / processing device.

[0055] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or may be source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk® or C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection to the external computer may be made (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may utilize state information of the computer readable program instructions to execute computer readable program instructions to personalize the electronic circuitry to implement aspects of the invention.

[0056] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0057] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, produce means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0058] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device, causing the computer, other programmable apparatus, or other device to perform a series of operational steps to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0059] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in an order other than that noted in the figures. For example, two blocks shown in succession may actually be accomplished as a single step that is performed simultaneously or nearly simultaneously or in a partially or fully overlapping manner, or the blocks may sometimes be performed in reverse order depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0060] It should be noted that any of the methods described herein may include the additional step of providing a system comprising individual software modules embodied in a computer-readable storage medium. The modules may include, for example, any or all of the components detailed herein. In this case, the method steps may be performed using individual software modules and / or sub-modules of such a system executed on the hardware processor 702. Furthermore, a computer program product may include a computer-readable storage medium having code adapted to be implemented to perform at least one method step described herein, including provisioning the individual software modules to the system.

[0061] In any case, it will be understood that the components illustrated herein may be implemented in various forms of hardware, software, or combinations thereof, such as, for example, application specific integrated circuits (ASICs), functional circuits, appropriately programmed digital computers with associated memories, etc. Given the teachings of the invention provided herein, one of ordinary skill in the relevant art will be able to contemplate other implementations of the components of the invention.

[0062] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings presented herein is not limited to a cloud computing environment, but rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0063] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0064] The characteristics are as follows:

[0065] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, on an as-needed basis, automatically and without the need for human interaction with the service provider.

[0066] Pervasive Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0067] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated depending on demand. Consumers generally do not have any control or knowledge over the exact location of the resources provided, although there is a sense of location independence in that they may be able to specify a higher level of abstraction location (e.g., country, state, or data center).

[0068] Rapid Elasticity: Capacity can be rapidly and elastically, in some cases automatically, provisioned to quickly scale out and released to quickly scale in. To the consumer, the capacity available for provisioning often appears infinite and can be purchased in any amount at any time.

[0069] Measured Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being utilized.

[0070] The service model is as follows:

[0071] Software as a Service (SaaS): The consumer is offered the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0072] Platform as a Service (PaaS): The capability offered to a consumer is to deploy applications created or acquired by the consumer, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, over the application hosting environment configuration.

[0073] Infrastructure as a Service (IaaS): The ability provided to a consumer is to provision processing, storage, network, and other underlying computing resources onto which the consumer can deploy and run software of their choice, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and in some cases, limited control over limited networking components (e.g., host firewalls).

[0074] The deployment models are as follows:

[0075] Private Cloud: The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may reside on-premise or off-premise.

[0076] Community Cloud: The cloud infrastructure is shared by several organizations to support a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by the organization or a third party and may reside on-premise or off-premise.

[0077] Public Cloud: The cloud infrastructure is made available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0078] Hybrid Cloud: A cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are held together by standardized or proprietary technologies that allow portability of data and applications (e.g., cloud bursting for load balancing between clouds).

[0079] Cloud computing environments are service-oriented and their focus is on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0080] Referring now to FIG. 8, an illustrative cloud computing environment 850 is depicted. As shown, the cloud computing environment 850 includes one or more cloud computing nodes 810 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or cellular phone 854A, a desktop computer 854B, a laptop computer 854C, or an automobile computer system 854N, or combinations thereof, may communicate. The nodes 810 may communicate with each other. They may be physically or virtually grouped in one or more networks, such as a private, community, public, or hybrid cloud, or combinations thereof, as previously described (not shown). This enables the cloud computing environment 850 to provide infrastructure, platform, and / or software as a service without the cloud consumer having to maintain resources on a local computing device. It will be understood that the types of computing devices 854A-N shown in FIG. 8 are illustrative only, and that computing node 810 and cloud computing environment 850 can communicate with any type of computerized device via any type of network and / or network-addressable connections (e.g., using a web browser).

[0081] Referring now to Figure 9, a set of functional abstraction layers provided by cloud computing environment 850 (Figure 8) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 9 are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0082] Hardware and software layer 960 includes hardware and software components. Examples of hardware components include mainframes 961, RISC (reduced instruction set computing) architecture based servers 962, servers 963, blade servers 964, storage devices 965, and networks and networking components 966. In some embodiments, software components include network application server software 967 and database software 968.

[0083] The virtualization layer 970 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 971, virtual storage 972, virtual networks including virtual private networks 973, virtual applications and operating systems 974, and virtual clients 975. In one example, the management layer 980 may provide the following functions: Resource provisioning 981 provides dynamic procurement of computing and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing 982 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources.

[0084] In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection of data and other resources. User portal 983 provides consumers and system administrators with access to the cloud computing environment. Service level management 984 provides cloud computing resource allocation and management such that required service levels are met. Service level agreement (SLA) planning and achievement 985 provides pre-arrangement and procurement of cloud computing resources where future requirements are anticipated according to SLAs.

[0085] The workload layer 990 provides examples of functions for which cloud computing environments are utilized. Examples of workloads and functions provided from this layer include mapping and navigation 991, software development and lifecycle management 992, virtual classroom instructional delivery 993, data analytics processing 994, transaction processing 995, and artificial intelligence algorithms (including threshold detection and floating point calculations) processing 996, in accordance with one or more embodiments of the present invention.

[0086] The terminology used herein is merely for the purpose of describing particular embodiments and is not intended to be limiting of the present invention. In this specification, the singular forms "a", "an" and "the" are intended to include the plural unless the context clearly indicates otherwise. Furthermore, it will be understood that the terms "comprises" and / or "comprising", when used herein, specify the presence of stated features, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of other features, steps, operations, elements, components, or groups thereof, or combinations thereof.

[0087] At least one embodiment of the present invention may provide beneficial effects, such as, for example, a framework (e.g., a set of one or more framework configurations) that replaces complex manual (e.g., custom-built) development of model recovery logic. As illustratively described herein, the framework is configured and instantiated with a set of fault detection components and an associated model recovery pipeline. Once instantiated, the framework plugs into a given lifecycle using logs as input and delivers new model artifacts for the new model version into the existing lifecycle pipeline. In one or more illustrative embodiments, the framework is a cloud-based framework and platform for end-to-end development and lifecycle management of AI applications.

[0088] Although the description of various embodiments of the present invention has been presented for illustrative purposes, the description is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements over the art found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A system comprising: a floating-point calculation unit configured to perform a dot product operation according to the first floating-point value and the second floating-point value; detection logic operably coupled to the floating point calculation unit; Equipped with the detection logic is configured to calculate a difference between a fixed-point sum of exponents of the first floating-point value and the second floating-point value and detect the presence of a condition prior to completion of the dot product operation by the floating-point computation unit based on the calculated difference; responsive to detecting the existence of the condition, the detection logic is further configured to cause the floating point computation unit to avoid performing a subset of computations that would normally be performed as part of the dot product operation.

2. 2. The system of claim 1, wherein the condition detected is whether a result of the calculation of the difference between a fixed-point sum of exponents of the first floating-point value and the second floating-point value is less than a threshold value.

3. The system of claim 2 , wherein the threshold value is one of a positive value and zero.

4. 3. The system of claim 2, wherein the existence of the detected condition serves as a predictor that the avoided subset of calculations will result in a dot product of the first floating-point value and the second floating-point value being less than the threshold.

5. 5. The system of claim 1, further comprising a voltage scaling controller operatively coupled to the detection logic and configured to reduce an operating voltage of the detection logic in proportion to a processing delay associated with the floating point calculation unit.

6. 6. The system of claim 1, wherein the dot product operation is part of a training stage for a neural network model used in an artificial intelligence system.

7. 7. A system according to any preceding claim implemented as part of one or more integrated circuits.

8. 1. A method comprising: calculating a difference between a fixed-point sum of the exponents of the first floating-point value and the second floating-point value; detecting the existence of a condition based on the calculated difference prior to completion of a dot product operation by a floating-point calculation unit configured to perform a dot product operation according to the first floating-point value and the second floating-point value; in response to detecting the existence of the condition, causing the floating point computation unit to avoid performing a subset of computations that would normally be performed as part of the dot product operation; Including, A method wherein one or more of said steps are performed by a processing circuit configured to execute instruction code.

9. 9. The method of claim 8, wherein the condition detected is whether a result of the calculation of the difference between a fixed-point sum of exponents of the first floating-point value and the second floating-point value is less than a threshold value.

10. The method of claim 9 , wherein the threshold value is one of a positive value and zero.

11. 10. The method of claim 9, wherein the existence of the detected condition serves as a predictor that the avoided subset of calculations will result in a dot product of the first floating-point value and the second floating-point value being less than the threshold.

12. the difference between a fixed-point sum of the exponents of the first floating-point value and the second floating-point value is [0010] and ex ｉ represents the exponent of one of the first floating-point value and the second floating-point value; ey ｉ represents the exponent of the other of the first floating-point value and the second floating-point value, N represents the number of components constituting the exponent, and S ｐ represents a group of positive products, and S ｎ The method of claim 9 , wherein represents a group of negative products.

13. 13. A method according to any of claims 8 to 12, wherein the dot product operation is part of a training stage for a neural network model used in an artificial intelligence system.

14. 1. An apparatus comprising: At least one processor; at least one memory containing instruction code; Equipped with 14. Apparatus, wherein said at least one memory and said instruction code, together with said at least one processor, are configured to cause said apparatus to perform the steps of any of claims 8 to 13.

15. A computer program product causing a computer to carry out the steps according to any one of claims 8 to 13.

16. 1. A system comprising: a rectified linear unit; a floating-point computation unit operatively coupled to the rectified linear unit and configured to perform a dot product operation according to the first floating-point value and the second floating-point value; detection logic operably coupled to the floating point calculation unit; Equipped with the detection logic is configured to detect the presence of a condition prior to completion of the dot product operation by the floating-point computation unit by calculating a difference between a fixed-point sum of exponents of the first floating-point value and the second floating-point value and, based on the calculated difference, comparing the calculated difference to a threshold; in response to the calculated difference being less than the threshold, the detection logic is further configured to cause the floating point computation unit to avoid performing a subset of computations that would normally be performed as part of the dot product operation and to cause an output of the floating point computation unit to be controlled by the rectifying linear unit.

17. 20. The system of claim 16 implemented as part of an artificial intelligence system.

18. 20. The system of claim 16 implemented as part of one or more integrated circuits.

Images