Apparatus and method for configuring a neural network

Abstract

An apparatus and method for configuring a neural network are provided, the apparatus comprising a processing unit to combine respective low-part values of two weight values, one weight value selected from a first set of weight values of a plurality of non-uniformly distributed weight values of the neural network, the other weight value selected from a second set of weight values of the plurality of non-uniformly distributed weight values of the neural network, and concurrently apply a first set of multipliers of the neural network to the set of combined low-part values and a second set of multipliers of the neural network to a set of non-zero high-part values, each high-part value associated with a low-part value, wherein both the low-part values and the high-part values are from a common weight value.

Description

BACKGROUND

[0001] Some embodiments described in this disclosure relate to computerized devices that execute neural networks, and more particularly, but not exclusively, to computerized devices that execute neural networks having fixed-point weight values.

[0002] The term neural network is generally used to describe computer systems inspired by the human brain and nervous system. Neural networks generally involve a large number of parallel operations and processing objects arranged and connected in layers (or tiers). The term “depth” in deep neural network (DNN) refers to the number of layers in such a neural network. The term “inference” refers to applying parameters and computations from a trained neural network model to infer one or more output values in response to one or more input values. A typical computation in a neural network layer involves summing a number of products between layer input values (also referred to as activation values) and associated weight values, and mapping the resulting sum to a layer output value. A neural network has a plurality of weight values for such computations. In some neural networks, the plurality of weight values are a plurality of fixed-point values. Fixed-point representation of a value is a method of representing an approximation of a real value, where there is a designated quantity of bits after a radix point. In decimal notation, the radix point is referred to as a decimal point. In binary notation, the radix point can be referred to as a “binary point”. In binary notation, the number “one quarter” can be represented as 0.01 in base 2.

[0003] A fixed-point value represented by a designated quantity of binary bits (denoted by n) can have one of 2n values. A fixed-point value can be divided into two parts, a high part value consisting of its designated quantity of most significant bits, and a low part value consisting of all least significant bits of the fixed-point value that are not members of the high part value. For example, a 16-bit binary value can be divided into a high part value having 5 bits and a low part value having 11 bits. In another example, a 16-bit binary value can be divided into high and low part values of equal size, each having 8 bits.

[0004] When a plurality of fixed-point values has a uniform distribution, for at least some of the plurality of fixed-point values, any of the n bits can be non-zero. However, there are DNNs in which the plurality of weight values of the DNN has a non-uniform distribution with small variance around a designated value and a long tail. Some examples of non-uniform distributions are normal distribution, Gaussian distribution, Laplace distribution, and Cauchy distribution. When the small variance is around a 0 value, and each of the plurality of weight values is represented by a fixed-point value, there is a designated quantity of most significant bits denoted by k, for which most of the plurality of weight values have a high part value equal to zero. SUMMARY

[0005] Some embodiments described in the present invention use a non-uniform distribution of multiple weight values of a neural network to improve the computational efficiency of the neural network execution. In these embodiments, the amount of multiplication performed by the neural network is reduced by using a multiplier to multiply an activation value by two or more low-part values of two or more weight values of the multiple weight values and multiply the activation value by only non-zero high-part values of the two or more weight values.

[0006] According to a first aspect of the present invention, an apparatus for configuring a neural network comprises a processing unit configured to: receive a first set of weight values and a second set of weight values of a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low-part value and a high-part value; generate a set of combined low-part values, each combined low-part value generated by combining a respective low-part value of two weight values, one weight value selected from the first set and the other weight value selected from the second set; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low-part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high-part values of at least some of the first set and the second set, each high-part value of the set of high-part values associated with a low-part value, wherein both the low-part value and the high-part value are from a common weight value of the first set and the second set. Concurrently applying the second set of multipliers to the set of high-part values and the first set of multipliers to the set of combined low-part values reduces the amount of cycles required to compute the output value compared to non-concurrent computation, which helps to reduce the operational cost of the neural network, thereby reducing the power consumption, and in addition reduces the implementation cost of the neural network by reducing the amount of multipliers compared to providing a multiplier for each weight value of the plurality of weight values, without adversely affecting the accuracy of the output value.

[0007] According to a second aspect of the application, a method for configuring a neural network comprises: receiving a first set of weight values and a second set of weight values from a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; generating a set of combined low part values, each combined low part value being generated by combining respective low part values of two weight values, one weight value being selected from the first set and the other weight value being selected from the second set; configuring the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values being associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set.

[0008] According to a third aspect of the application, an apparatus for executing a neural network comprises a processing unit configured to: receive a first set of weight values and a second set of weight values from a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; generate a set of combined low part values, each combined low part value being generated by combining respective low part values of two weight values, one weight value being selected from the first set and the other weight value being selected from the second set; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values being associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set; receive an input value; compute the one or more output values in response to the input value.

[0009] According to a fourth aspect of the present application, a software program product for configuring a neural network comprises: a non-transitory computer readable storage medium; first program instructions for receiving a first set of weight values and a second set of weight values from a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; second program instructions for generating a set of combined low part values, each combined low part value being generated by combining respective low part values of two weight values, one weight value being selected from the first set and the other weight value being selected from the second set; third program instructions for configuring the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values being associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set. Optionally, the first program instructions, the second program instructions and the third program instructions are executed by at least one computerized processor from the non-transitory computer readable storage medium.

[0010] According to a fifth aspect of the present application, a computer program comprises program instructions that, when executed by a processor, cause the processor to: receive a first set of weight values and a second set of weight values from a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; generate a set of combined low part values, each combined low part value being generated by combining respective low part values of two weight values, one weight value being selected from the first set and the other weight value being selected from the second set; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values being associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set.

[0011] In an implementation form of the first aspect and the second aspect, each combined low part value in the set of combined low part values is associated with one activation value in the set of activation values. Optionally, applying the first set of multipliers to the set of combined low part values comprises each multiplier in the first set of multipliers multiplying one combined low part value in the set of combined low part values by a respective activation value associated therewith. Optionally, applying the second set of multipliers to the set of high part values comprises each multiplier in the second set of multipliers multiplying one high part value in the set of high part values by a respective activation value associated with a respective low part value associated with the high part value. Optionally, calculating the at least one output value comprises calculating the at least one output value using the first set of intermediate values and the second set of intermediate values. Optionally, each high part value in the set of high part values is not equal to zero. Optionally, each respective activation value multiplied by a high part value in the set of high part values is not equal to zero. Compared to the implementation form in which multipliers are also applied to zero values, only applying the multipliers of the second set of multipliers to high part values not equal to zero, and additionally or alternatively to activation values not equal to zero, helps to reduce the amount of multipliers in the second set of multipliers, thus reducing the implementation cost of the neural network without affecting the accuracy of the output values of the neural network inference.

[0012] In an implementation form of the first aspect and the second aspect, the neural network is configured to compute the at least one output value further comprises that at least some of the multipliers selected from one or more of the first set of multipliers and the second set of multipliers compute another second set of intermediate values by multiplying one of a set of other high part values of at least some other high part values selected from one or more of the first set and the second set by the respective activation value, wherein the other high part value is associated with an other low part value, the other low part value and the other high part value both derived from another common weight value of the first set and the second set, wherein the respective activation value is associated with a respective other low part value, the respective other low part value being associated with the other high part value. Optionally, computing the at least one output value comprises further using the another second set of intermediate values to compute the at least one output value. Optionally, the second set of multipliers has an identified number of multipliers. Optionally, the identified number of multipliers is 32. Optionally, a set of non-zero high parts resulting from selecting the full set of high part values of the first set and the second set comprises more high part values than the identified number of multipliers, the set of other high part values comprising a number of other high part values of the full set of high part values that are not members of the set of high part values. Optionally, receiving the first set comprises receiving a first sequence of low part values of the first set; receiving a first sequence of bits in order, each bit being associated with a low part value of the first sequence of low part values and having a value of 1 when a respective high part value associated with the low part value is not equal to zero, and otherwise having a value of 0; receiving a first sequence of high part values in order, each high part value being associated with a non-zero bit of the first sequence of bits; wherein receiving the second set comprises receiving a second sequence of low part values of the second set; receiving a second sequence of bits in order, each bit being associated with an other low part value of the second sequence of low part values and having a value of 1 when another respective high part value associated with the other low part value is not equal to zero, and otherwise having a value of 0; receiving a second sequence of high part values in order, each high part value being associated with another non-zero bit of the second sequence of bits. Compared to some other approaches, e.g. using ordinal numbers or providing a high part value for each low part value, using the sequences of bits to associate each high part value in the sequence of high part values with a low part value in the sequence of low part values reduces the amount of memory needed to create such an association.

[0013] In an implementation form of the first aspect and the second aspect, each weight value comprises a weight bit quantity; each low part value comprises a low part bit quantity; and each low part value is a least significant part of the weight value. Optionally, the low part bit quantity is half of the weight bit quantity. Optionally, the weight bit quantity is selected from a group of bit quantities consisting of 4, 8, 16, 32 and 64.

[0014] In an implementation form of the first aspect and the second aspect, the neural network comprises a plurality of layers, each layer having a plurality of layer weight values of the plurality of weight values of the neural network. Optionally, the first set and the second set are selected from the plurality of layer weight values of one layer of the plurality of layers. Selecting the first set and the second set from the plurality of weight values of one layer of the plurality of layers supports considering the distribution of the plurality of weight values of the layer when selecting the first set and the second set, thereby reducing the amount of cycles required to calculate the output value of the neural network.

[0015] In an implementation form of the first aspect and the second aspect, the first pair of sets comprises the first set and the second set. Optionally, the second pair of sets comprises another first set of the plurality of weight values and another second set of the plurality of weight values. Optionally, the processing unit is further configured to: receive the second pair of sets; generate another set of combined low part values, each combined low part value being generated by combining respective other low part values of two other weight values, one weight value being selected from the another first set and the other weight value being selected from the another second set; configure the neural network to compute the at least one output value by further concurrently computing: another first set of intermediate values by applying another first set of multipliers of the neural network to the another set of combined low part values; another second set of intermediate values by applying another second set of multipliers of the neural network to another set of high part values of at least some of the another first set and the another second set, each other high part value of the another set of high part values being associated with an other low part value, wherein the other low part value and the other high part value both originate from another common weight value of the another first set and the another second set; wherein the first set of multipliers is different from the another first set of multipliers and the second set of multipliers is different from the another second set of multipliers. Optionally, the first set of multipliers is different from the another first set of multipliers and the second set of multipliers is different from the another second set of multipliers. Using the another first set of multipliers different from the first set of multipliers and the another second set of multipliers different from the second set of multipliers, computing the first set of intermediate values and the second set of intermediate values by concurrently computing the another first set of intermediate values and the another second set of intermediate values, facilitates to improve throughput of the neural network.

[0016] In an implementation form of the first aspect and the second aspect, the first set and the second set are generated by receiving another set of the plurality of weight values and dividing the another set of the plurality of weight values into the first set and the second set, such that an amount of weight values of the first set is equal to an amount of weight values of the second set.

[0017] In an implementation form of the first aspect and the second aspect, the plurality of weight values has a non-uniform distribution with a variance smaller than a variance threshold identified.

[0018] Other systems, methods, features, and advantages of the application will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the application, and be protected by the accompanying claims.

[0019] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments belong. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of the embodiments, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be limiting. BRIEF DESCRIPTION OF DRAWINGS

[0020] Some embodiments are herein described, by way of example only, with the understanding that the with the accompanying drawings the details as to how the embodiments can be practiced are set forth in part in the description below, and in part are obvious from the description of the drawings in conjunction with the detailed description below, and like reference numerals refer to like elements throughout.

[0021] In the drawings:

[0022] Figure 1 is a schematic diagram representing an exemplary product of two numbers represented in binary;

[0023] Figure 2 is a schematic block diagram representing a portion of an exemplary neural network provided by some embodiments;

[0024] Figure 3 is a schematic block diagram of an exemplary apparatus provided by some embodiments;

[0025] Figure 4 is a flowchart schematically representing an optional operational flow for configuring a neural network provided by some embodiments;

[0026] Figure 5 is a flowchart schematically representing another optional operational flow for configuring a neural network provided by some embodiments;

[0027] Figure 6 is a flowchart schematically representing an optional operational flow for performing a neural network provided by some embodiments. DETAILED DESCRIPTION

[0028] For brevity, the term "non-uniform distribution around a value" is used to mean "non-uniform distribution with small variance around the value," and the term "non-uniform multiple values around an identified value" is used to mean "multiple values with non-uniform distribution with small variance around the identified value." Thus, "non-uniform multiple values around 0" is used to mean "multiple values with non-uniform distribution with small variance around 0."

[0029] It is noted that the following description focuses on embodiments that include a neural network with non-uniform multiple weight values around 0. However, the scope of the apparatus and methods described herein is not limited to these embodiments. In other possible embodiments, the multiple weight values are non-uniform multiple weight values around another value, and an offset value can be added to each of the multiple weight values to map the multiple weight values to other non-uniform multiple weight values around 0. The offset value can be a negative or non-negative value.

[0030] A typical deep neural network includes millions of parameters and can require millions of arithmetic operations, resulting in required computational and digital memory resources exceeding the capabilities of many devices, such as mobile devices, some embedded devices, and some custom hardware devices. Moreover, the amount of computation cycles required for inference affects both the amount of time required for inference, thereby affecting the throughput of the neural network, and the amount of power consumed by the neural network, thereby affecting the cost of operation of the neural network.

[0031] One method of reducing the production cost of a neural network is by reducing the amount of physical computational resources of the neural network, some examples being reducing the amount of computational elements of the neural network and the amount of memory used by the neural network. For example, when the neural network is implemented as an integrated circuit, reducing the amount of physical computational resources of the neural network reduces the area of the semiconductor package that includes the integrated circuit.

[0032] Some neural networks reduce the amount of computational resources by reducing the amount of bits used to store at least some of the multiple weights of the neural network. This practice is referred to as quantization of the neural network. Quantization can also include reducing the amount of bits used to store at least some of the activation values of the neural network. However, quantizing a neural network can reduce the accuracy of the output of the neural network. In quantization, the original multiple weight values (each weight value represented by an original amount of bits) can be mapped to a multiple of quantized weight values (each quantized weight value represented by a reduced amount of bits), where the reduced amount of bits is less than the original amount of bits. Since fewer bits are used to represent each quantized weight value, there are fewer possible values for the quantized weight values than for the weight values, and more than one of the original multiple weight values can be mapped to a common quantized weight value. Thus, the accuracy of the neural network using the multiple of quantized weight values can be reduced compared to using the original multiple weight values.

[0033] Some neural networks use layer computational resources to compute in multiple batches (iterations) rather than concurrently applying the layer computational resources to all of the weight values of the layer. In such schemes, in each batch, the layer computational resources are applied to some of the multiple weights of the layer, and the output of the layer is computed using the multiple batch results. The amount of time required to compute the output of such a neural network increases as a function of the amount of batches performed, reducing throughput and possibly increasing the amount of power required to perform the neural network.

[0034] The product of a multiplication by zero is zero. When a weight value has a high part value equal to zero, the product computed using such a weight value also has at least some high part equal to zero. Referring now to Figure 1 , a diagram 100 is shown that represents an exemplary product of two numbers in binary representation. In this example, the number N1 is equal to 3 and has a high part HP1 equal to zero, and the number N2 is equal to 3 and has a high part HP2 equal to zero. In this example, the product P1 is equal to 9, which is the result of the number N1 multiplied by the number N2. In this example, the product P1 has a high part HP3 equal to zero. It is noted that the high part does not necessarily include all of the contiguous most significant bits equal to zero. For example, in the diagram 100, the number N1 and the number N2 both have other bits contiguous to the high parts HP1 and HP2, respectively, that are zero, but they are excluded from the respective high parts.

[0035] When multiple weight values of a neural network have a non-uniform distribution around zero, there can be a number of most significant bits k for which the multiple weight values each have a high part equal to zero, and thus multiple products computed using the multiple weight values each have a high part equal to zero. Optionally, the multiple weight values are multiple fixed point values. Optionally, the multiple weight values are multiple fixed point quantized values generated by quantizing multiple floating point weight values of the neural network.

[0036] Some embodiments described herein propose using this property to reduce the amount of explicit multiplication by reducing the amount of computational resources of the neural network. To this end, in some embodiments described herein, a first set of multipliers of the neural network is applied to a set of combined low-part values, each combined low-part value being produced by combining low-part values of two or more of a plurality of weight values, while concurrently applying a second set of multipliers of the neural network to a set of non-zero high-part values. Optionally, the second set of multipliers has fewer multipliers than the first set of multipliers. Optionally, the set of combined low-part values is produced by combining respective low-part values of two weight values, one weight value being selected from a first set of the plurality of weight values and the other weight value being selected from a second set of the plurality of weight values. Optionally, the set of non-zero high-part values is at least a portion of a set of high-part values of at least some of the first set and the second set, such that each high-part value of the set of high-part values is associated with a low-part value, wherein the low-part value and the high-part value are derived from the first set and the second set. Optionally, one or more intermediate values computed by applying the first set of multipliers and the second set of multipliers are used to compute one or more output values of the neural network. Avoiding applying a multiplier to a high-part that is equal to zero can reduce the amount of multipliers of the neural network, thereby reducing the production cost and the operational cost of the neural network. Applying one multiplier to a combined low-part value facilitates representing non-zero parts of two or more weight values using fewer bits, while maintaining the full precision of the neural network. Furthermore, applying one multiplier to a combined low-part value facilitates reducing the amount of computation cycles required to compute two or more products of two or more low-part values, thereby reducing the operational cost of the neural network by reducing power consumption.

[0037] It is important to note that the above method does not quantize the plurality of weights of the neural network. According to some embodiments of the present application, the full precision of the neural network is preserved, as there is no loss of significant values. Multiplying by zero produces zero, and therefore avoiding applying a multiplier to a high-part that is equal to zero does not lose significant values. In such embodiments, when the second set of multipliers has fewer multipliers than the first set of multipliers, the amount of computational resources of the neural network is reduced without affecting the precision of the output values inferred by the neural network. Furthermore, applying fewer multipliers can reduce the power consumption of the neural network.

[0038] Optionally, applying a multiplier to a high-part value comprises multiplying the high-part value by an associated activation value. Optionally, applying the second set of multipliers to the set of non-zero high-part values, the non-zero high-part values being associated with each activation value of a respective set of non-zero activation values. When the associated activation value is zero, avoiding multiplying the high-part value by the associated activation value facilitates reducing the amount of computation cycles used by the neural network, thereby reducing the power consumption of the neural network.

[0039] Applying the first set of multipliers concurrently to the set of combined low- part values and applying the second set of multipliers to at least some of the set of high-part values facilitates computing the output values using less physical computing resources, thereby reducing production and implementation costs of the neural network without reducing throughput of the neural network.

[0040] The first set of weight values and the second set of weight values can have a quantity of weight values having non-zero high parts that exceeds a quantity of multipliers in the second set of multipliers. Optionally, at least some of the first set of multipliers and the second set of multipliers are applied to a set of other high-part values to compute one or more other intermediate results. Optionally, the one or more other intermediate results are further used to compute one or more output values of the neural network. Applying at least some of the multipliers to the set of other high-part values can maintain accuracy of the neural network output without increasing a quantity of physical computing resources thereof.

[0041] Optionally, one or more of the second set of multipliers are applied to one or more combined high-part values, each combined high-part value resulting from combining two or more of the set of high-part values. Applying one or more of the second set of multipliers to the one or more combined high-part values facilitates applying the second set of multipliers to at least some of the set of high-part values, thereby enabling computation of the one or more output values without applying at least some of the multipliers to a set of other high-part values, thereby reducing computation time and thus increasing throughput and reducing power consumption to execute the neural network.

[0042] Before at least one embodiment is described in detail, it is to be understood that embodiments are not necessarily limited to a detailed description of the construction and arrangement of the components and / or methods described herein and / or illustrated in the drawings and / or examples. The implementations described herein support other embodiments and / or uses and / or modifications as are apparent from a clear understanding of the application, which is delineated in the following detailed description.

[0043] Embodiments can be a system, a method, and / or a computer program product. The computer program product can include one or more computer readable storage media having computer readable program instructions stored therein that cause a processor to perform various aspects of the embodiments.

[0044] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing. Computer readable storage media used in the disclosure can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any suitable combination of the foregoing. The disclosure is not limited to the above examples of computer readable storage media, but can include any computer readable storage media that can be used to store and access the data and / or instructions used in the disclosure.

[0045] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0046] Computer readable program instructions for carrying out operations of embodiments can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), and the connection can be made using an

[0047] Aspects of the embodiments are provided herein with reference to the drawing figures, which are described below. The drawings are included to provide a thorough understanding of the embodiments. A person of ordinary skill in the art will recognize that the applications described with reference to the drawings can be practiced without resorting to the details illustrated and described in the drawing figures and accompanying text.

[0048] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0049] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0050] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0051] Reference is now made to Figure 2 , which shows a schematic block diagram of a portion of an example neural network 200, according to some embodiments. In these embodiments, a first set of weight values 210 includes weight values 210-1 through 210-N, and a second set of weight values 220 includes weight values 220-1 through 220-N. Optionally, the first set 210 and the second set 220 include weight values of a plurality of weight values of the neural network. Optionally, each weight value of the first set 210 and the second set 220 has a low portion and a high portion. For example, weight values 210-1, 210-2, and 210-N can each have low portion values 211-1, 211-2, and 211-N, respectively, and high portion values 212-1, 212-3, and 212-N, respectively. Similarly, in this example, weight values 220-1, 220-2, and 220-N can each have low portion values 221-1, 221-2, and 221-N, respectively, and high portion values 222-1, 222-3, and 222-N, respectively. Optionally, each low portion value of a weight value is a least significant portion of the weight value.

[0052] Optionally, each weight value of the plurality of weight values includes a weight bit quantity, some examples being 8 bits and 32 bits. Other examples of weight bit quantities include 4, 16, and 64. Optionally, each low portion value includes a low portion bit quantity, which is less than the weight bit quantity. Optionally, the low portion bit quantity is half of the weight bit quantity.

[0053] Optionally, the first set 210 and the second set 220 are used to produce a set 230 of combined low part values. For example, a combined low part value 230-1 can be produced by combining the low part value 211-1 of the weight value 210-1 of the first set 210 and the low part value 221-1 of the weight value 220-1 of the second set 220. Similarly, a combined low part value 230-2 can be produced by combining the low part value 211-2 of the weight value 210-2 of the first set 210 and the low part value 221-2 of the weight value 220-2 of the second set 220. Similarly, a combined low part value 230-N can be produced by combining the low part value 211-N of the weight value 210-N of the first set 210 and the low part value 221-N of the weight value 220-N of the second set 220.

[0054] Optionally, each of the combined low part values in the set 230 of combined low part values is associated with an activation value in a set 240 of activation values. For example, the combined low part value 230-1 can be associated with an activation value 240-1 of the set 240 of activation values. Similarly, the combined low part value 230-2 can be associated with an activation value 240-2 of the set 240 of activation values, and the combined low part value 230-N can be associated with an activation value 240-N of the set 240 of activation values.

[0055] Optionally, a first set 260 of multipliers, including multipliers 260-1 through 260-N, is applied to the set 230 of combined low part values to optionally multiply each of the combined low part values in the set 230 of combined low part values by an activation value of the set 240 of activation values with which it is associated. For example, the multiplier 260-1 can be used to multiply the combined low part value 230-1 by the activation value 240-1. Similarly, the multiplier 260-2 can be used to multiply the combined low part value 230-2 by the activation value 240-2, and the multiplier 260-N can be used to multiply the combined low part value 230-N by the activation value 240-N.

[0056] Optionally, the set 250 of high part values includes a plurality of high part values of at least some of the first and second sets, such as the high part value 212-1 of the weight value 210-1 of the first set 210, the high part value 212-N of the weight value 210-N of the first set 210, and the high part value 222-N of the weight value 220-N of the second set 220. Optionally, the set 250 of high part values includes all of the corresponding high part values of all of the weight values of the first set 210 and the second set 220. Optionally, each of the high part values in the set 250 of high part values is not equal to zero.

[0057] Optionally, a second set of multipliers 270, including multipliers 270-1 through 270-M, is applied to the set of high-part values 250 to optionally multiply each high-part value in the set of high-part values 250 by an activation value of the set of activation values 240 associated therewith, the activation value being associated with a low-part value. In the present example, high-part value 212-1 is associated with low-part value 211-1, both of which are derived from weight value 210-1. In the present example, low-part value 211-1 is associated with activation value 240-1, and thus, in the present example, high-part value 212-1 is associated with activation value 240-1. Similarly, in the present example, high-part value 212-N and high-part value 222-N are both associated with activation value 240-N.

[0058] It is noted that one activation value can be provided to more than one multiplier in the second set of multipliers 270. In the present example, activation value 240-N is provided to multiplier 270-2 to optionally multiply high-part value 212-N, and to multiplier 270-M to optionally multiply high-part value 222-N. Optionally, second multiplier 270-1 is applied to high-part value 212-1 and activation value 240-1.

[0059] Optionally, some high-part values, such as high-part value 212-2, high-part value 222-1, and high-part value 222-2, are not members of the set of high-part values 250 and are not provided to the second set of multipliers 270. Optionally, high-part value 212-2, high-part value 222-1, and high-part value 222-2 each equal zero. Optionally, high-part value 212-2 is non-zero, and the associated activation value 240-2 equals zero, and thus is not provided to one of the multipliers in the second set of multipliers 270.

[0060] Optionally, the first set of multipliers 260 computes a first set of intermediate values. Optionally, the second set of multipliers 270 computes a second set of intermediate values. Optionally, using the first set of intermediate values and the second set of intermediate values, one or more output values are computed, optionally using one or more adders 280.

[0061] Optionally, each intermediate value in the first set of intermediate values has a respective low portion and a respective high portion. For example, one intermediate value in the first set of intermediate values can result from multiplying the combined low portion value 230-1 (resulting from combining low portion value 211-1 and low portion value 221-1) by the respective application value 240-1. Optionally, one or more respective low portion values of the first set of intermediate values are provided to a first adder of the one or more adders 280, and one or more respective high portion values of the first set of intermediate values are provided to a second adder of the one or more adders 280. For example, a respective low portion of the output of multiplier 260-1 can be provided to a first adder of adders 280, and a respective high portion of the output of multiplier 260-1 can be provided to a second adder of adders 280. Similarly, another respective low portion of the output of multiplier 260-N can be provided to a first adder of adders 280, and another respective high portion of the output of multiplier 260-N can be provided to a second adder of adders 280.

[0062] In some embodiments, a neural network comprising portion 200 uses the following exemplary device configuration.

[0063] Now referring also to Figure 3 , a schematic block diagram of an exemplary device 300 is shown. In one embodiment, the device is a single apparatus. In another embodiment, the device is a system of two or more apparatuses (e.g., computers) that interact to implement the functionality described herein. Optionally, device 300 comprises a processing unit 301 connected to a memory 302. Processing unit 301 can be any type of programmable or non-programmable circuitry for performing the operations described herein. Processing unit 301 can comprise hardware and software. For example, processing unit 301 can comprise one or more processors and a transitory or non-transitory memory carrying a program that, when executed by the one or more processors, causes the processing unit to perform the respective operations. Memory 302 can be a transitory or non-transitory memory. Optionally, memory 302 carries a program. Optionally, memory 302 is a non-volatile digital memory, some examples being a hard disk drive, a solid state drive, a network memory, and a storage network. Optionally, processing unit 301 retrieves a plurality of weight values of a neural network from memory 302. Optionally, processing unit 301 is connected to a digital communication network interface 303. Optionally, processing unit 301 is connected to memory 302 through digital communication network interface 303.

[0064] To configure a neural network, in some embodiments, device 300 implements the following optional method.

[0065] Now referring also to Figure 4, a flow chart schematically illustrating an optional operational process 400 for configuring a neural network provided in some embodiments. In these embodiments, at 401, processing unit 301 receives a first set 210 and a second set 220 of a plurality of weight values ​​of a neural network. Optionally, the neural network is executed by processing unit 301. Optionally, the neural network is executed by another processing unit (not shown) connected to processing unit 301.

[0066] Optionally, the plurality of weight values ​​have a non-uniform distribution with a variance less than an identified variance threshold. Optionally, the variance threshold is identified such that at least one identified portion of the plurality of weight values ​​has a corresponding high portion equal to zero, for example, at least half of the plurality of weight values ​​may have a corresponding high portion equal to zero. In another example, the variance threshold is identified such that at least one third of the plurality of weight values ​​have a corresponding high portion equal to zero. Reference is now also made to Figure 5 , a flow chart schematically illustrating another optional operational flow 500 for configuring a neural network provided by some embodiments is shown. In these embodiments, receiving the first set 210 at 401 includes the processing unit 301 receiving a first sequence of low partial values ​​of the first set 210 at 501, such as low partial values ​​211-1 through 211-N. Optionally, at 502, the processing unit 301 sequentially receives a first sequence of bits, each bit associated with a low partial value in the first sequence of low partial values, and having a value of 1 when the corresponding high partial value associated with the low partial value is not zero, and having a value of 0 otherwise. For example, when high partial value 212-1 and high partial value 212-N are not zero, the first sequence of bits in positions 1 and N has a value of 1. When high partial value 212-2 is zero, the first sequence of bits in position 2 has a value of 0. Optionally, at 503, the processing unit 301 sequentially receives a first sequence of high partial values, each high partial value associated with a non-zero bit of the first sequence. Continuing with the above example, the first high partial value sequence includes high partial value 212-1 and subsequent high partial value 212-N, where high partial value 212-2 is not a member of the first high partial value sequence. It should be noted that high partial value 212-N follows high partial value 212-1 in the first high partial value sequence, but does not necessarily follow high partial value 212-1. Optionally, high partial value set 250 is a combination of the first high partial value sequence and the second high partial value sequence.

[0067] Optionally, receiving the second set 220 includes the processing unit 301 receiving, in 511, a second sequence of low partial values of the second set 220, e.g., low partial values 221-1 to 221-N. Optionally, in 512, the processing unit 301 receives, in order, a second sequence of bits, each bit being associated with one other low partial value of the second sequence of low partial values, and the value of the second sequence of bits being 1 when another respective high partial value associated with the other low partial value is not equal to zero, and otherwise being 0. Optionally, in 513, the processing unit 301 receives, in order, a second sequence of high partial values, each high partial value being associated with another non-zero bit of the second sequence of bits.

[0068] Now referring again to Figure 4 .

[0069] Optionally, the first set 210 and the second set 220 are generated by the processing unit 301 receiving another set of weight values of the plurality of weight values, and the processing unit 301 dividing the another set of weight values into the first set 210 and the second set 220 such that the amount of weight values (denoted by N) of the first set 210 is equal to the amount of weight values of the second set 220.

[0070] Optionally, in 402, the processing unit 301 generates a set 230 of combined low partial values. Optionally, each combined low partial value of the set 230 of combined low partial values is generated by combining respective low partial values of two weight values, one weight value being selected from the first set 210 and the other weight value being selected from the second set 220. Optionally, each combined low partial value of the set 230 of combined low partial values is generated by combining respective low partial values of more than two weight values, e.g., one weight value being selected from the first set 210, another weight value being selected from the second set 220, and yet another weight value being selected from a third set of low partial values not shown.

[0071] In 403, the processing unit 301 optionally configures a neural network to compute one or more output values. Optionally, the neural network includes a plurality of layers, each layer having a plurality of layer weight values of the plurality of weight values of the neural network. Optionally, the first set 210 and the second set 220 are selected from the plurality of layer weight values of one layer of the plurality of layers. Optionally, the one or more output values are one or more output values of the layer.

[0072] Optionally, computing the one or more output values is by concurrently computing, in 411, a first set of intermediate values and computing, in 412, a second set of intermediate values. Optionally, computing the one or more output values in 403 includes computing, in 420, the one or more output values using the first set of intermediate values and the second set of intermediate values.

[0073] Optionally, each combined low partial value in the set of combined low partial values 230 is associated with one activation value in the set of activation values 240.

[0074] Optionally, the processing unit 301 configures the neural network to compute, in 411, a first set of intermediate values by applying the first set of multipliers 260 to the set of combined low partial values 230. Optionally, applying the first set of multipliers 260 to the set of combined low partial values 230 includes each multiplier in the first set of multipliers 260 multiplying one combined low partial value in the set of combined low partial values 230 by a respective activation value associated therewith.

[0075] Optionally, the processing unit 301 configures the neural network to compute, in 412, a second set of intermediate values by applying the second set of multipliers 270 to the set of high partial values 250. Optionally, the set of high partial values 250 is a set of high partial values of at least some of the first set 210 and the second set 220. Optionally, each high partial value of the set of high partial values 250 is associated with a low partial value, wherein the low partial value and the high partial value originate from a common weight value of the first set and the second set. Optionally, applying the second set of multipliers 270 to the set of high partial values 250 includes each multiplier in the second set of multipliers 270 multiplying one high partial value in the set of high partial values 250 by a respective activation value associated with a respective low partial value associated with the high partial value. Optionally, each respective activation value multiplied by one high partial value in the set of high partial values 250 is not equal to zero. Optionally, applying the second set of multipliers 270 to the set of high partial values 250 includes at least one multiplier in the second set of multipliers 270 multiplying an activation value by a combined high set value resulting from combining two high partial values in the set of high partial values 250 respectively associated with the activation value. Optionally, the combined high set value results from combining more than two high partial values.

[0076] Optionally, the neural network includes additional multipliers not in either of the first set of multipliers 260 and the second set of multipliers 270.

[0077] Optionally, the second set of multipliers 270 has a number of multipliers identified by M, for example 32 multipliers. Optionally, the set of non-zero high partials resulting from selecting the full set of high partial values in the first set 210 and the second set 220 includes more than M high partial values. Optionally, the set of high partial values 250 includes M high partial values. Optionally, the set of other high partial values includes a number of other high partial values in the full set of high partial values that are not members of the set of high partial values 250, for example one or more of high partial value 212-2, high partial value 222-1, and high partial value 222-2.

[0078] Optionally, in 430, the processing unit 301 computes another second set of intermediate values. Optionally, the processing unit 301 computes the another second set of intermediate values using at least some of the multipliers selected from the first set of multipliers 260 and the second set of multipliers 270. Optionally, each of the at least some of the multipliers is applied to one of a set of other high partial values, e.g., high partial value 212-2, of at least some other high partial values selected from one or more of the first set 210 and the second set 220. Optionally, the other high partial value is associated with an other low partial value, e.g., low partial value 211-2, where both the other low partial value and the other high partial value originate from another common value, e.g., weight value 210-2, of the first set 210 and the second set 220. Optionally, applying one of the at least some of the multipliers to one of the set of other high partial values includes multiplying the other high partial value by a respective activation value, e.g., 240-2, associated with a respective other low partial value, e.g., low partial value 211-2, associated with the other high partial value, e.g., high partial value 212-2. In 420, the processing unit 301 optionally further computes the one or more output values using the another second set of intermediate values.

[0079] Optionally, the plurality of weight values includes a plurality of pairs of sets, and the above-described method is repeated, optionally concurrently, for other pairs using other sets of multipliers of the neural network. Optionally, the first pair of sets includes the first set 210 and the second set 220. Optionally, another pair of sets includes another first set of the plurality of weight values and another second set of the plurality of weight values.

[0080] Optionally, 401, 402, 403, 411, 412, and 420 are repeated using another pair of sets and another first set of multipliers of the neural network and another second set of multipliers of the neural network. Optionally, the first set of multipliers is different from the another first set of multipliers. Optionally, the second set of multipliers is different from the another second set of multipliers. Optionally, 401, 402, 403, 411, 412, and 420 are repeated using another pair of sets and the first set of multipliers of the neural network and the second set of multipliers of the neural network.

[0081] Optionally, the method 400 is applied to each of one or more layers of the neural network.

[0082] In some embodiments, the computer program includes program instructions which, when executed by the processing unit 301, cause the apparatus 300 to implement the method 400.

[0083] In some embodiments, the apparatus 300 executes the neural network. In some such embodiments, the apparatus implements the following optional method.

[0084] Reference is now also made to Figure 6 Fig. 6 shows a flowchart schematically representing an optional operational procedure 600 for performing a neural network, according to some embodiments. In these embodiments, in 601, the processing unit 301 optionally configures the neural network by performing the method 400 to optionally compute one or more output values. In 610, the processing unit 301 optionally receives an input value, and in 620, the processing unit 301 optionally computes one or more output values in response to the input value.

[0085] The description of the various embodiments is merely meant to be illustrative and not exhaustive or limiting to the disclosed embodiments. Many modifications and variations to the described embodiments will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The language used in the specification is expressly intended to be used in a descriptive sense and not as a limiting factor. The terms used in the specification are not intended to be used as limiting terms. The selection of terms used in this document is made to best explain the principles, practical applications or technical improvements over the technology found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0086] It is expected that many related neural networks will be developed in the future, and the scope of the term neural network is intended to include all such new technologies a priori.

[0087] The term "about" as used herein means 10%.

[0088] The terms "comprising," "having," and their synonyms mean "including, but not limited to." This term encompasses the terms "consisting of" and "consisting essentially of."

[0089] The phrase "consisting essentially of" means that the composition or method can include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0090] As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. For example, the terms "a compound" or "at least one compound" can include a plurality of compounds, including mixtures thereof.

[0091] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0092] The word "optional" is used herein to mean "being an option, and not a requirement." Any particular implementation of an embodiment can include a plurality of "optional" features, unless such features are incompatible with each other.

[0093] In this application, various embodiments can be presented through a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the embodiments. Accordingly, the description of a range should be considered to have specifically disclosed the end points of the ranges and every number and amount within that range. For example, a description of a range of from 1 to 6 should be considered to have specifically disclosed the end points of 1 and 6, as well as any number between 1 and 6, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0094] When a range of numbers is indicated herein, any number (fraction or integer) within the indicated range is included. The phrases "range between a first indicated number and a second indicated number" and "range from a first indicated number to a second indicated number" are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractions and integers therebetween.

[0095] It should be appreciated that certain features of the embodiments, which are, for brevity, described in the context of separate embodiments, can also be provided in combination in a single embodiment. Conversely, various features of the embodiments, which are, for brevity, described in the context of a single embodiment, can also be provided separately or in any suitable

[0096] Although the embodiments have been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations as fall within the spirit and broad scope of the appended claims.

[0097] Herein, all publications, patents and patent applications referred to in this specification are incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present embodiments. To the extent that section headings are used, they should not be construed as necessarily limiting the subject matter described.

Claims

1. An apparatus for configuring a neural network, the apparatus comprising: comprising a processing unit to: receive a first set and a second set of a plurality of weight values of the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; produce a set of combined low part values, each combined low part value produced by combining respective low part values of two weight values, one weight value selected from the first set and the other weight value selected from the second set; each combined low part value of the set of combined low part values associated with one activation value of a set of activation values; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set; wherein applying the first set of multipliers to the set of combined low part values includes each multiplier of the first set of multipliers multiplying one combined low part value of the set of combined low part values by a respective activation value associated therewith; wherein applying the second set of multipliers to the set of high part values includes each multiplier of the second set of multipliers multiplying one high part value of the set of high part values by a respective activation value associated with a respective low part value associated with the high part value; wherein computing the at least one output value includes computing the at least one output value using the first set of intermediate values and the second set of intermediate values.

2. The apparatus of claim 1, wherein, each high part value of the set of high part values is not equal to zero.

3. The apparatus of claim 1 or 2, wherein, each respective activation value multiplied by a high part value of the set of high part values is not equal to zero.

4. The apparatus of claim 1, wherein, configuring the neural network to compute the at least one output value further includes: at least some multipliers selected from one or more of the first set of multipliers and the second set of multipliers compute another second set of intermediate values by multiplying one other high part value of a set of at least some other high part values of other high part values from one or more of the first set and the second set by the respective activation value, wherein the other high part value is associated with an other low part value, both the other low part value and the other high part value derived from another common weight value of the first set and the second set, wherein the respective activation value is associated with a respective other low part value associated with the other high part value; wherein computing the at least one output value includes further computing the at least one output value using the another second set of intermediate values.

5. The apparatus of claim 4, wherein, the second set of multipliers has an identified quantity of multipliers; wherein a set of non-zero high part values resulting from selecting all of the set of complete high part values in the first set and the second set includes more high part values than the identified quantity of multipliers; wherein the set of other high part values includes a plurality of other high part values in the set of complete high part values that are not members of the set of high part values.

6. The apparatus of claim 5, wherein: receiving the first set includes: receiving a first low part value sequence of the first set; receiving a first bit sequence in order, each bit associated with a low part value in the first low part value sequence and having a value of 1 when a respective high part value associated with the low part value is not equal to zero and otherwise having a value of 0; receiving a first high part value sequence in order, each high part value associated with a non-zero bit of the first bit sequence; wherein receiving the second set includes: receiving a second low part value sequence of the second set; receiving a second bit sequence in order, each bit associated with an other low part value in the second low part value sequence and having a value of 1 when another respective high part value associated with the other low part value is not equal to zero and otherwise having a value of 0; receiving a second high part value sequence in order, each high part value associated with another non-zero bit of the second bit sequence.

7. The apparatus of claim 5 or 6, wherein, the identified quantity of multipliers is 32.

8. The apparatus of claim 1, wherein: each weight value includes a quantity of weight bits; each low part value includes a quantity of low part bits; each low part value is a least significant portion of the weight value.

9. The apparatus of claim 8, wherein, the quantity of low part bits is half of the quantity of weight bits.

10. The apparatus of claim 8 or 9, wherein, the quantity of weight bits is selected from a group of quantities of bits consisting of 4, 8, 16, 32, and 64.

11. The apparatus of claim 1, wherein, the neural network includes a plurality of layers, each layer having a plurality of layer weight values of the plurality of weight values of the neural network; wherein the first set and the second set are selected from the plurality of layer weight values of one of the plurality of layers.

12. The apparatus of claim 1, wherein: a first pair of sets includes the first set and the second set; wherein a second pair of sets includes another first set of the plurality of weight values and another second set of the plurality of weight values; wherein the processing unit is further to: receive the second pair of sets; generate another set of combined low part values, each combined low part value generated by combining respective other low part values of two other weight values, one weight value selected from the another first set and another weight value selected from the another second set; configure the neural network to compute the at least one output value by further concurrently computing: another first set of intermediate values by applying another first set of multipliers of the neural network to the another set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high partial values of at least some of the first set and the second set, each other high partial value of the set of other high partial values being associated with an other low partial value, wherein the other low partial value and the other high partial value are both derived from a common weight value of the first set and the second set; wherein the first set of multipliers is different from the second set of multipliers.

13. The apparatus of claim 1, wherein, the first set and the second set are generated by: receiving a set of other weight values of the plurality of weight values; dividing the set of other weight values into the first set and the second set such that an amount of weight values of the first set is equal to an amount of weight values of the second set.

14. The apparatus of claim 1, wherein, the plurality of weight values have a non-uniform distribution with a variance less than a variance threshold.

15. A method for configuring a neural network, the method comprising: comprising: receiving a first set and a second set of a plurality of weight values from the neural network, wherein each weight value of the first set and the second set has a low partial value and a high partial value; generating a set of combined low partial values, each combined low partial value being generated by combining respective low partial values of two weight values, one weight value being selected from the first set and the other weight value being selected from the second set; each combined low partial value of the set of combined low partial values being associated with one activation value of a set of activation values; configuring the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low partial values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high partial values of at least some of the first set and the second set, each high partial value of the set of high partial values being associated with a low partial value, wherein the low partial value and the high partial value are both from a common weight value of the first set and the second set; wherein applying the first set of multipliers to the set of combined low partial values includes each multiplier of the first set of multipliers multiplying one combined low partial value of the set of combined low partial values by a respective activation value associated therewith; wherein applying the second set of multipliers to the set of high partial values includes each multiplier of the second set of multipliers multiplying one high partial value of the set of high partial values by a respective activation value associated with a respective low partial value, the respective low partial value being associated with the high partial value; wherein computing the at least one output value includes computing the at least one output value using the first set of intermediate values and the second set of intermediate values.

16. An apparatus for performing a neural network, the apparatus comprising: comprising a processing unit to: configure the neural network by: receive a first set and a second set of a plurality of weight values from the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; generate a set of combined low part values, each combined low part value generated by combining respective low part values of two weight values, one weight value selected from the first set and the other weight value selected from the second set; each combined low part value of the set of combined low part values is associated with one activation value of a set of activation values; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high part values of at least some of the first set and the second set, each high part value of the set of high part values associated with a low part value, wherein both the low part value and the high part value are from a common weight value of the first set and the second set; receive an input value; compute the one or more output values in response to the input value; wherein applying the first set of multipliers to the set of combined low part values includes each multiplier of the first set of multipliers multiplying one combined low part value of the set of combined low part values by a respective activation value associated therewith; wherein applying the second set of multipliers to the set of high part values includes each multiplier of the second set of multipliers multiplying one high part value of the set of high part values by a respective activation value associated with a respective low part value associated with the high part value; wherein computing the at least one output value includes computing the at least one output value using the first set of intermediate values and the second set of intermediate values.

17. A software program product for configuring a neural network, characterized in that comprise: a non-transitory computer readable storage medium; first program instructions to receive a first set and a second set of a plurality of weight values from the neural network, wherein each weight value of the first set and the second set has a low part value and a high part value; second program instructions to generate a set of combined low part values, each combined low part value generated by combining respective low part values of two weight values, one weight value selected from the first set and the other weight value selected from the second set; each combined low part value of the set of combined low part values is associated with one activation value of a set of activation values; third program instructions to configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high-part values of at least some of the first set and the second set, each high-part value of the set of high-part values being associated with a low-part value, wherein the low-part value and the high-part value are both from a common weight value of the first set and the second set; wherein the first program instructions, the second program instructions, and the third program instructions are executed by at least one computerized processor from the non-transitory computer-readable storage medium; wherein applying the first set of multipliers to the set of combined low-part values includes each multiplier of the first set of multipliers multiplying one combined low-part value of the set of combined low-part values by a respective activation value associated therewith; wherein applying the second set of multipliers to the set of high-part values includes each multiplier of the second set of multipliers multiplying one high-part value of the set of high-part values by a respective activation value associated with a respective low-part value associated with the high-part value; wherein computing the at least one output value includes computing the at least one output value using the first set of intermediate values and the second set of intermediate values.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium has computer-readable program instructions that, when executed by a processor, cause the processor to: receive a first set and a second set of a plurality of weight values from a neural network, wherein each weight value of the first set and the second set has a low-part value and a high-part value; produce a set of combined low-part values, each combined low-part value produced by combining respective low-part values of two weight values, one weight value selected from the first set and the other weight value selected from the second set; each combined low-part value of the set of combined low-part values being associated with one activation value of a set of activation values; configure the neural network to compute at least one output value by concurrently computing: a first set of intermediate values by applying a first set of multipliers of the neural network to the set of combined low-part values; a second set of intermediate values by applying a second set of multipliers of the neural network to a set of high-part values of at least some of the first set and the second set, each high-part value of the set of high-part values being associated with a low-part value, wherein the low-part value and the high-part value are both from a common weight value of the first set and the second set; wherein applying the first set of multipliers to the set of combined low-part values includes each multiplier of the first set of multipliers multiplying one combined low-part value of the set of combined low-part values by a respective activation value associated therewith; wherein applying the second set of multipliers to the set of high-part values includes each multiplier in the second set of multipliers multiplying one high-part value in the set of high-part values by a respective activation value associated with a respective low-part value, the respective low-part value being associated with the high-part value; wherein calculating the at least one output value includes calculating the at least one output value using the first set of intermediate values and the second set of intermediate values.

Images