Execution unit, processing unit, and method for approximating a function

Abstract

The execution unit is configured to receive a first computer program instruction that populates multiple entries into a lookup table, each entry containing an x ​​value representing a point on the curve of the function and a corresponding y value, where consecutive entries among the multiple entries form an interval of the function, receive a second computer program instruction that populates multiple entries into a lookup table stored in local memory associated with the execution unit and indicates the input value, determine the interval of the function to which the input value belongs based on the lookup table, and generate an output value corresponding to the input value by interpolating between the y values ​​of the entries that form the interval.

Description

Technical Field

[0001] The present disclosure relates to an execution unit, a processing apparatus, and a method for approximating a function.

Background Art

[0002] Neural networks are used in the fields of machine learning and artificial intelligence. A neural network includes an arrangement of a set of nodes interconnected by links and interacting with each other. The principle of neural network computation is inspired by information on how electrical stimuli transmit information in the human brain. For this reason, the nodes are generally called neurons. When a neural network is regarded as a computational graph, the nodes may also be called vertices, and the links between the nodes may be called edges. The network can obtain input data, and specific nodes perform operations on the data. The results of these operations are passed to other nodes. The output of each node is called its activation or node value. Each link is associated with a weight. The weights define the connectivity between the nodes of the neural network. Various techniques are known that enable a neural network to learn by changing the values of the weights.

[0003] Figure 1A shows a highly simplified version of one arrangement of nodes in a neural network to illustrate a specific related concept. This type of configuration is commonly used in learning or training and includes an input layer of nodes, one or more hidden layers of nodes, and an output layer of nodes. In practice, each layer has many nodes, and currently there are typically many hidden layers, and current deep learning models for image and language processing contain a large number of layers. Each node in the input layer Ni can generate an activation or node value at its output, which is produced by executing a nonlinear function or activation function on the data provided to that node. The vector of node values ​​from the input layer is scaled by the respective parameter or weight vectors at the input of each node in the hidden layer, and each weight defines the connectivity between that particular node and its connected nodes in the hidden layer. In practice, since networks can have millions or billions of nodes and can be connected multidimensionally, the vectors are often tensors. Node N hThe weights applied to the input are labeled w0···w2. In the example in Figure 1A, each node in the input layer is connected to each node in the hidden layer, at least initially. Each node in the hidden layer can run an activation function on the data provided to it, and similarly generate the output vectors supplied to each node N0 in the output layer N0. This is an example of a fully connected architecture or feedforward network (K. Hornik, M. Stinchcombe, and H. White, "Multilayer Feedforward Networks are Universal Approximators," Neural Networks, vol.2, pp.359-366, 1989), a convolutional architecture (Y. LeCun, K. Kavukcuoglu, and C. Farabet, "Convolutional networks and applications in vision," Proceedings of the 2010 IEEE International Symposium on Circuits and Systems, Paris, France, pp.253-256, 2010), and a transformer model including a self-attention layer (A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, AN Gomez, L. Kaiser, and I. Polosukhin, "Attention is All You Need," arXiv preprint, arXiv:1706.03762). Various other types of architectures have been developed over the years, including [cs.CL], 2017).

[0004] Each node weights its incoming data, for example, by performing a dot product of the node's input activation and its intrinsic weight for each inflow link. An activation function is then executed on the weighted data. The activation function can be, for example, a sigmoid function or a hyperbolic tangent function. See Figure 1B. Recently used activation functions include the rectifier linear unit (ReLU) and the Gaussian error linear unit (GeLU). The network learns by acting on the data input in the input layer, assigning weights to the activations from each node, weighting the data input to each node in the hidden layer, and executing the activation function. Thus, the nodes in the hidden layer act on the weighted data and supply outputs to the nodes in the output layer. The nodes in the output layer can also assign weights to their edges used to generate output node values. While there are different learning approaches, in each case there is forward propagation through the network from left to right in Figure 1A, calculation of the loss (e.g., cross-entropy loss), and backpropagation through the network from right to left in Figure 1A, of the loss gradient. In the next cycle, each node updates its network parameters, taking into account the backpropagated gradient, and generates a modified set of weights. In this way, the network can be trained to perform its desired behavior (Y. LeCun, L. Bottou, G.B. Orr, and K.-R. Muller, "Efficient BackProp," Neural Networks: Tricks of the Trade, Springer, 2012). [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] When processing large datasets using neural networks, it is desirable to approximate functions. For example, it may be necessary to approximate activation functions. Certain calculations involving the use of floating-point numbers may be performed by processing units that execute software instructions. Such calculations include those of known functions such as hyperbolic or logistic functions. These functions can be computationally very complex. In other situations, for example, it may be desirable to sample random numbers according to a given probability distribution function to facilitate Monte Carlo sampling.

[0006] English Patent Application GB2580138A describes a technique for approximating a function in hardware using a set of lookup tables (hereinafter referred to as LUTs) containing entries for values ​​of the function within a specific range of input values, which are referred to herein as the “interval” of the function. The function is divided into multiple ranges, in which a numerical approximation is applied and in which one or more LUTs are used. This means that the LUTs are pre-stored and the specific function to be approximated must be defined in advance. [Means for solving the problem]

[0007] According to the first aspect of this disclosure, A first computer program instruction is received to populate a lookup table with multiple entries, each entry containing an x ​​value representing a point on the curve of the function and a corresponding y value, and consecutive entries of the multiple entries form an interval of the function. The plurality of entries are entered into a lookup table stored in the local memory associated with the execution unit. Upon receiving a second computer program instruction indicating an input value, Based on the lookup table, the interval of the function to which the input value belongs is determined. An execution unit is provided that is configured to interpolate between the y values ​​of entries forming the interval to generate an output value corresponding to the input value.

[0008] The execution unit may be configured to perform linear interpolation between the y values ​​of the entries that form the interval.

[0009] The first computer program instruction may have a field that holds an instruction for a memory address that stores a plurality of entries. The instruction may be an address in data memory that stores the plurality of entries, or a pointer to a register that stores an address in data memory, and the address stored in data memory stores the plurality of entries.

[0010] The multiple entries may be ordered by their x values. The execution unit may be configured to compare the input value with the x values ​​in the lookup table. The execution unit may be configured to sequentially scan the x values ​​in the lookup table and compare the input value with the x values. The initial x value may be the smallest x value, and the execution unit may be configured to sequentially scan the x values ​​from the smallest x value to the largest x value. The execution unit may be configured to determine the interval as an interval including a first x value greater than the input value and the x value immediately preceding the first x value. Alternatively, the initial value may be the largest x value, and the execution unit may be configured to sequentially scan the x values ​​from the largest x value to the smallest x value. The execution unit may be configured to determine the interval as an interval including a first x value smaller than the input value and the x value immediately following the first x value.

[0011] The lookup table may have a storage capacity greater than the number of entries in the plurality of entries. The execution unit may be configured to terminate the comparison of input values ​​when it reaches the last entry of the plurality of entries stored in the lookup table. The execution unit may be configured to execute a second computer program instruction by performing a predetermined maximum number of comparisons and / or interpolations, the predetermined maximum number corresponding to the number of comparisons and / or interpolations required to reach the last entry of the plurality of entries stored in the lookup table. The execution unit may be configured to perform K comparisons and / or interpolations per execution cycle. The execution unit may be configured to execute a maximum number of execution cycles, the maximum number of execution cycles corresponding to the maximum number of comparisons and / or interpolations divided by K. The execution unit may be configured to access a comparison register that stores the maximum number of execution cycles or the maximum number of comparisons and / or interpolations. The execution unit may be configured to execute a third computer program instruction and store the maximum number in the comparison register.

[0012] The execution unit may be configured to set the size of the lookup table based on the operands of the first computer program instruction or configuration values ​​associated with the first computer program instruction.

[0013] The second computer program instruction may indicate an input vector containing multiple input values. The execution unit may be configured to determine, for each input value of the input vector, the interval of the function to which each input value belongs, based on the lookup table. The execution unit may be configured to generate output values ​​corresponding to each input value by interpolating between the y values ​​that form each interval. The input vector may contain one of two, four, or eight input values.

[0014] The execution unit may include a mapping register for storing indications of the input values ​​of the input vector from which output values ​​have been generated. The execution unit can output an output vector containing each output value when the mapping register indicates that output values ​​have been generated for all input values. The mapping register may include a number of bits equal to the number of input values ​​in the input vector. The execution unit can set the bits of the mapping register in response to the generation of output values ​​for input values ​​at the positions of the input vector corresponding to the bits.

[0015] The execution unit may be configured to determine that the input value is outside the domain of the function. The domain of the function may be defined by the minimum and maximum x values ​​of the lookup table. In response that the input value is less than the minimum x value, the execution unit may be configured to produce an output value corresponding to the y value of the minimum x value. In response that the input value is greater than the maximum x value, the execution unit may be configured to produce an output value corresponding to the y value of the maximum x value. In response that the input value is outside the domain, the execution unit may produce an output value corresponding to zero or NaN.

[0016] According to a second aspect of this disclosure, a processing unit is provided which includes the execution unit described in the first aspect. The processing unit may include local memory. The processing unit may also be a tile processor.

[0017] The processing unit may be configured to execute a supervisor thread and worker threads. The first computer program instruction may be executed as part of the supervisor thread. The second computer program instruction may be executed as part of the worker thread. The processing unit may be configured to execute a computer program including the first computer program instruction and the second computer program instruction. The first computer program instruction may form part of the execution preamble of the computer program.

[0018] A third aspect of this disclosure provides a processing apparatus including the processing unit described in the second aspect. The processing apparatus may include a plurality of processing units. At least one, preferably all, of the processing units may include the execution unit described in the first aspect. The processing units can communicate via an exchange fabric that implements time-deterministic exchange.

[0019] According to a fourth aspect of this disclosure, a method performed by an execution unit, A step of receiving a first computer program instruction to input multiple entries into a lookup table, wherein each entry includes an x ​​value representing a point on the curve of a function and a corresponding y value, and consecutive entries of the multiple entries form an interval of the function. The steps include: inputting the multiple entries into a lookup table stored in local memory associated with the execution unit; The steps include receiving a second computer program instruction indicating an input value, The steps include determining the interval of the function to which the input value belongs based on the lookup table, A method is provided which includes the step of interpolating between the y values ​​of entries forming the interval to generate an output value corresponding to the input value.

[0020] Further optional features of the method of the fourth aspect are described above in connection with the first, second, and third aspects and can be combined in any combination.

[0021] According to a fifth aspect, A first computer program instruction that, when executed by an execution unit, causes the execution unit to input a plurality of entries into a lookup table stored in a local memory associated with the execution unit, each entry including an x value representing a point on a curve of a function and a corresponding y value, and consecutive entries of the plurality of entries form an interval of the function, and / or A second computer program instruction indicating an input value that, when executed by the execution unit, causes the execution unit to Determine an interval of the function to which the input value belongs based on the lookup table, Interpolate between the y values of the entries forming the interval to generate an output value corresponding to the input value, and a computer program product is provided that includes the second computer program instruction.

[0022] Further optional features of the method of the fourth aspect and the computer program product of the fifth aspect are described above in connection with the first, second, and third aspects and can be combined in any combination.

[0023] The present disclosure also extends to a computer program product including instructions that, when executed, cause an execution unit to execute any of the methods described herein. The present disclosure also extends to a non-transitory computer-readable medium that, when executed, stores instructions that cause an execution unit to execute any of the methods described herein.

[0024] For a better understanding of the present invention and to more clearly show how the present invention is implemented, the following will be described by way of example only, with reference to the accompanying drawings.

Brief Description of the Drawings

[0025] [Figure 1A] This is a very simplified schematic diagram of a neural network. [Figure 1B] This is a very simplified schematic diagram of a neuron. [Figure 2A] An example of this disclosure is a schematic block diagram of a processor in which an example is implemented. [Figure 2B] An example of a processor chip in which this disclosure is implemented is shown in a schematic diagram. [Figure 3] This is an illustration of an example function. [Figure 4] This is an example of a lookup table. [Figure 5] This is a schematic block diagram of an example execution unit. [Figure 6] This is a schematic flowchart of an example method executed in the execution unit.

[0026] In drawings, corresponding reference numerals indicate corresponding components. Those skilled in the art will understand that elements in drawings are shown for conciseness and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in drawings may be exaggerated relative to others to facilitate understanding of various examples. Also, common and well-understood elements that are useful or necessary in commercially viable examples are often omitted to make these various examples easier to understand. [Modes for carrying out the invention]

[0027] Generally, the examples in this disclosure provide instructions that can be executed by an execution unit, enabling programmers to input into a lookup table (LUT). These “populate” instructions allow the programmer to store values ​​that define intervals of a function in the LUT. The LUT is held in memory associated with the execution unit (e.g., scratch memory). The populate instructions for storing values ​​in the LUT may be executed, for example, by a supervisor thread executed by the execution unit as part of the execution preamble.

[0028] The examples of this disclosure further provide instructions for interpolating the output values ​​of received input values ​​based on values ​​in a LUT. Thus, the execution unit can quickly and efficiently calculate values, for example, to approximate an arbitrary function.

[0029] Furthermore, by providing these instructions, programmers can generate values ​​that appropriately approximate the relevant function over a desired range of expected input values. Therefore, users with relevant knowledge of the values ​​most likely to be expected in the context of the application can configure the LUT accordingly. Thus, large LUTs that accommodate a very wide range of input values ​​can be avoided.

[0030] An example is implemented in a processing unit which can take the form of processor 4, which will be described in more detail with reference to Figure 2A. In some examples, processor 4 can take the form of a tile in a multi-tile processing unit. Examples of such multi-tile processing units are described in more detail in the prior application, U.S. Patent Application US2020 / 0319861A1, which is incorporated by reference.

[0031] Referring to Figure 2A, an example of a processor 4 is shown, including details of the execution unit 18 and context register 26. The illustrated processor 4 includes a weight register file 26W and can therefore be particularly suited to machine learning applications in which machine learning models are trained by tuning the weights of those models. However, the examples of the present invention are not limited to machine learning applications and are more broadly applicable. Furthermore, the described processor 4 is a multithreaded processor capable of executing M threads simultaneously. The processor 4 can support the execution of M worker threads and one supervisor thread, where the worker threads perform arithmetic operations on data to produce results, and the supervisor thread coordinates the worker threads and controls the synchronization and send / receive functions of the processor 4.

[0032] The processor 4 includes instruction buffers 53 for each of the M threads that can run concurrently. The context registers 26 include main register files (MRFs) 26M for each of the M worker contexts and supervisor contexts. The context registers further include auxiliary register files (ARFs) 26A for at least each of the worker contexts. The context registers 26 further include a common weight register file (WRF) 26W that can be accessed and read by all currently running worker threads. The WRF can be associated with the supervisor context in that the supervisor thread is the only thread that can write to the WRF. The context registers 26 may also include groups of control state registers 26CSR for each of the supervisor context and worker contexts. The execution unit 18 includes a main execution unit 18M and an auxiliary execution unit 18A. The main execution unit 18M includes a load / storage unit (LSU) 55 and an integer arithmetic logic unit (IALU) 56. The auxiliary execution unit 18A includes at least a floating-point arithmetic unit (FPU).

[0033] In each of the J interleaved time slots S0...SJ-1, the scheduler controls a fetch stage (323, see Figure 5) to fetch at least one instruction for each thread from the instruction memory 11 into one of the J instruction buffers 53 corresponding to the current time slot. In this example, each time slot is one execution cycle of the processor, but other schemes are not excluded (e.g., weighted round robin). In each execution cycle of the processor 4 (i.e., each cycle of the processor clock that clocks the program counter), the fetch stage 323 fetches a single instruction or a small “instruction bundle” (e.g., a 2-instruction bundle or a 4-instruction bundle), depending on the implementation. Each instruction is then issued via a decode stage (324, see Figure 5) to either the LSU 55 or IALU 56 of the main execution unit 18M, or the FPU of the auxiliary execution unit 18A, depending on whether the instruction is a memory access instruction, an integer arithmetic instruction, or a floating-point arithmetic instruction, according to its opcode. The LSU55 and IALU56 of the main execution unit 18M execute their instructions using registers from the MRF26M, with specific registers in the MRF26M specified by the instruction operands. The FPU of the auxiliary execution unit 18A performs operations using registers in the ARF26A and WRF 26W, with specific registers in the ARF specified by the instruction operands. In the example, the registers in the WRF may be implicitly included in the instruction type (i.e., predetermined for that instruction type). The auxiliary execution unit 18A may also include circuitry in the form of internal logic latches for holding some internal states 57 for use when performing one or more operations of the floating-point arithmetic instruction types.

[0034] In an example of fetching and executing instructions within a bundle, the individual instructions within a given instruction bundle are executed simultaneously in parallel via independent pipelines 18M and 18A (shown in Figure 2A). In an example of executing a bundle of two instructions, the two instructions may be executed simultaneously along their respective auxiliary and main pipelines. In this case, the main pipeline is configured to execute instruction types that use MRF, and the auxiliary pipeline is used to execute instruction types that use ARF. Pairing instructions into appropriate complementary bundles may be handled by the compiler.

[0035] Each worker thread context has instances of the main register file (MRF) 26M and the auxiliary register file (ARF) 26A (i.e., there is one MRF and one ARF for each barrel thread slot). The functions described herein in relation to the MRF or ARF should be understood to operate on a per-context basis. However, there is a single shared weight register file (WRF) shared among threads. Each thread can access only the MRF and ARF of its own context 26. However, all currently running worker threads can access the common WRF. Thus, the WRF provides a common set of weights used by all worker threads. In the example, only the supervisor can write to the WRF, and only workers can read from the WRF.

[0036] The instruction set of processor 4 includes at least one type of load instruction, which, when executed, causes LSU 55 to load data from data memory 22 into each ARF 26A of the thread in which the load instruction was executed. The destination location within the ARF is specified by an operand of the load instruction. Another operand of the load instruction specifies an address register within each MRF 26M, which holds a pointer to the address of the data memory (322, see Figure 5) into which the data is loaded. The instruction set of processor 4 also includes at least one type of store instruction, which, when executed, causes LSU 55 to store data from each ARF of the thread in which the store instruction was executed into data memory 322. The source location of the store within the ARF is specified by an operand of the store instruction. Another operand of the store instruction specifies an address register within the MRF, which holds a pointer to the address in data memory 322 into which the data is stored. Generally, an instruction set may include separate types of load instructions and store instructions, and / or at least one type of load and store instruction that combines load and store operations into a single instruction.

[0037] In response to the opcode of the relevant type of arithmetic instruction, the arithmetic unit (e.g., FPU) within the auxiliary execution unit 18A performs the arithmetic operation specified by the opcode, which includes operating on the values ​​of the specified source registers in each ARF of the thread, and optionally on the values ​​of the source registers in the WRF. The result of the arithmetic operation is also output to the destination register of each ARF of the thread, which is explicitly specified by the destination operand of the arithmetic instruction.

[0038] The instruction set of processor 4 also includes the populate and interpolation instructions described herein.

[0039] The processor 4 may include an exchange interface 51 for exchanging data between the memory 11 and one or more other resources, such as other instances of the processor and / or external devices such as a network interface or network-attached storage (NAS) device. As described above, in the example, the processor 4 can form one of an array of interconnected processor tiles, each tile executing a portion of a broader program. Thus, individual processors 4 (tiles) form part of a broader processor or processing system 6. The tiles 4 can be connected to one another via interconnect subsystems, which are connected to each other via their respective exchange interfaces 51. The tiles 4 may be implemented on the same chip (i.e., die), on different chips, or in combination (i.e., the array may be formed from multiple chips, each containing multiple tiles 4). Thus, the interconnect system and the exchange interface 51 may include, accordingly, an internal (on-chip) interconnection mechanism and / or an external (inter-chip) exchange mechanism.

[0040] Figure 3 shows an exemplary function approximated using the techniques described herein. As shown in Figure 3, the curve 100 of a function f that maps input x values ​​to output y values ​​is represented by a series of n intervals 101-1, 101-2, ..., 101-n. Each interval is defined by two points 102 on the curve 100 that form a segment of the curve 100 and form the boundary of interval 101. Each point is represented by a horizontal and vertical coordinate (i.e., x and y coordinates). For example, interval 101-1 is bounded by points 102-1 and 102-2, interval 101-2 is bounded by points 102-2 and 102-3, and interval 101-n is bounded by points 102-n and 102-n+1. Thus, with the exception of the first point 102-1 and the last point 102-n+1, ​​each point acts as the upper bound of the preceding interval and the lower bound of the subsequent interval.

[0041] To map the input value 103 to the output value 104, we identify the x-coordinates of two points that define the interval to which the input belongs. Then, we extract the corresponding y-values. Next, we can calculate the output value by interpolating between the y-values. In the example shown in Figure 3, points 102-2 and 102-3 are the points that define the boundary of the interval 101-2, which contains the input value 103. Therefore, the y-values ​​of these points are used for interpolation.

[0042] In one example, the interpolation between points is linear. Therefore, the overall interpolation technique can be considered a piecewise linear interpolation technique. In such an example, the interpolation technique can consider only the y values ​​that define the boundaries of the identified intervals. For example, in the case shown in Figure 3, the y values ​​of points 102-2 and 102-3 may be sufficient to perform the interpolation.

[0043] However, in other examples, other interpolation techniques can be used. For example, polynomial interpolation techniques such as cubic interpolation or Hermitian interpolation can be used. In these and other examples, the interpolation technique can consider more of the points 102 on both sides of the interval in order to calculate the interpolated output value that belongs to the interval. For example, in the case shown in Figure 3, the y values ​​of points 102-1 and 102-4 can be considered. The interpolation can consider two, three, four, or any other number of points on both sides of the interval 101-2 containing the input value 103. The number of points considered on each side of the interval 101-2 does not have to be the same. For example, this may be the case when the determined interval lies between two points near the beginning or end of the function's domain.

[0044] Therefore, when we describe interpolation between two values ​​in this specification, we generally mean generating an output based on two values ​​and belonging to a range between those values. This does not mean that only two values ​​are used to calculate the output.

[0045] FIG. 4 shows a look-up table (LUT) 200 that stores data for performing the function approximation shown in FIG. 3. As shown, LUT 200 includes a plurality of entries that include x-values and corresponding y-values. The x1, y1 of LUT 200 correspond to the x-value and y-value of point 102-1, the x2, y2 correspond to the x-value and y-value of point 102-2, and so on. The entries within LUT 200 are ordered by their x-values such that x1 < x2, x2 < x3, …, xn < xn+1.

[0046] Next, a process for creating LUT 200 by execution unit 300 will be described in detail while referring to FIGS. 5 and 6. Execution unit 300 can correspond to execution unit 18 described above in relation to FIG. 2A. Elements of execution unit 18 / 300 that are not directly relevant to the present technology are omitted from FIG. 5 for simplicity and clarity.

[0047] As shown in FIG. 5, execution unit 300 is associated with a scratch memory 310 (also referred to as a “scratch pad” memory). Scratch memory 310 is a local memory that forms part of processing unit 4. Memory 310 may be fast and may be physically located near execution unit 300 or may be incorporated into the execution unit. In one example, memory 310 is a register associated with execution unit 300. For example, memory 310 may be context register 26 described above.

[0048] In step S601 of Figure 6, the execution unit 300 first receives a first instruction (hereinafter referred to as the "populate" instruction) for inputting the LUT 200. For example, the instruction memory 321 contains the populate instruction, which is then fetched by the fetch stage 323 and decoded by the decode stage 324. Next, the execution unit 300 executes the decoded instruction. The populate instruction does not have to be a specific instruction for inputting the LUT 200; instead, it may be a more general memory storage instruction that can store the LUT 200 in the scratch memory 310 when an appropriate operand or configuration value is provided.

[0049] The populate instruction includes an instruction for the location of entry 102 used to populate LUT200 as an operand. For example, the instruction may include an address in data memory 322, or a pointer to a register (not shown) that stores a memory address in data memory 322 containing point 102. In step S602, when the populate instruction is executed, the execution unit 300 loads point 102 from data memory 322 into LUT200.

[0050] Therefore, a programmer can configure LUT200 with values ​​representing arbitrary functions as part of a computer program executed by the execution unit 300. For example, a supervisor thread can issue a populate instruction. The instruction can form part of the execution preamble, i.e., the initial part of the program that sets values ​​for use in the main part of the program, which will be executed later by worker threads. In such an example, LUT200 can then be shared by multiple worker threads. However, in an alternative example, each worker thread can input and access its own LUT200.

[0051] The specific configuration of LUT200 may depend on the platform or implementation.

[0052] In some examples, the size of the LUT200 (i.e., the number of points that can be stored) may be determined in advance (i.e., during the manufacture or configuration of the processing unit). In such examples, a fixed, predetermined memory area 310 may be allocated to the LUT200 (and therefore to the entries in the LUT200). In such examples, not all of the allocated memory may be filled (i.e., not all of the entries in the LUT200 may be filled). Unfilled entries may be set to zero or NaN, or they may contain values ​​set by previous use of the memory 310.

[0053] In other examples, the size of LUT200 may be set by the programmer, for example, using operands or configuration data associated with the populate instruction. In such examples, the size of LUT200 is set to be equal to the number of entries to be stored.

[0054] The precision of the points may also be predetermined. Examples of point precisions include FP32, FP16, BF16, or FP8, but please understand that these are merely examples.

[0055] Next, in step S603, the execution unit 300 receives a second instruction based on the LUT200 to provide an output value corresponding to the received input value. This instruction may be referred to herein as an interpolation or "interp" instruction.

[0056] Interpolation instructions may include instructions for input values ​​as operands. For example, an instruction may include an address of data memory 322, or a pointer to a register (not shown) that stores the memory address of data memory 322 containing the input values. The input values ​​can take the form of a vector of input values.

[0057] Interpolation instructions may be issued by worker threads. Interpolation instructions can form part of a computer program, such as a computer program that trains a neural network.

[0058] For example, an interpolation command takes the following format: f <n> v <m>interp $aDest, $aSrc

[0059] Here, n represents the number of bits in the input value, and m represents the size of the input vector. Examples of n values ​​include 32 bits for full-precision floating-point values, 16 bits for half-precision floating-point values, and 8 bits for quarter-precision floating-point values. Examples of m values ​​include 2, 4, or 8. The operands $aDest and $aSrc represent the memory location where the instruction output is stored and the memory location where the input is stored, respectively.

[0060] In step S604, the execution unit 300 identifies the interval containing the input value for the first input value of the input vector. To identify the interval containing the input value, the execution unit 300 performs a sequential comparison between the input value and the x values ​​in the LUT 200, starting from the first entry x1. In other words, the execution unit 300 sequentially scans the x values ​​in the LUT 200. Since the x values ​​in the LUT are ordered by increasing the input value, the comparison ends when it is determined that the first x value is greater than the input value. Next, the interval between the first x value xi, which is greater than the input value, and the preceding x value in the LUT 200, xi-1, is recognized as the interval containing the input value.

[0061] In some examples, the execution unit 300 can alternatively sequentially scan x1 from the last entry (xn+1) down to gradually decreasing x values. In this example, the comparison ends when the first x value is found to be less than the input value. The interval containing the input value is then scanned, and the input value x i The interval between the first x-value xi, which is smaller than the given value, and the subsequent x-value xi+1 in LUT200 is recognized as the interval containing the input value. This substantially corresponds to a conceptually sequential scan from right to left, in contrast to the sequential scan from left to right of LUT200 as described above.

[0062] In step S605, the execution unit 300 calculates the y values ​​corresponding to the x values ​​that form the interval. i-1 , y i The y-values ​​are extracted. Next, the execution unit 300 interpolates between the y-values ​​to generate output values ​​corresponding to the input values. The interpolation may be linear interpolation or any other interpolation technique, including those described above.

[0063] In some situations, the first input value is equal to the x value. In one example, the execution unit 300 outputs the corresponding y value, and interpolation is not required. However, this approach may require an additional comparison not only to whether the input value is greater than (or less than in the alternative example) the x value, but also whether it is equal to the x value, to accommodate relatively rare cases. Therefore, in some situations, the execution unit 300 may instead not check whether the input value is equal to the x value, but instead simply apply the relevant interpolation technique. In such cases, the comparison during the sequential scanning process may be whether the x value is greater than or equal to (or less than or equal to in the alternative example) the x value.

[0064] Steps S604 and S605 may be performed for each subsequent input value in the input vector. In one example, steps S604 and S605 may be repeated sequentially for each input value in the input vector.

[0065] However, in another example, step S604 may first be performed for each value of the input vector. That is, each input value is compared with x1, then with x2, and so on, to identify the interval corresponding to each input value. This approach can minimize the number of times the memory storing the LUT is accessed. The comparison of specific input values ​​stops when an interval is identified. Interpolation S605 is then performed for each input value in relation to each identified interval.

[0066] Once all values ​​in the input vector are mapped to their corresponding output values, the execution unit 300 outputs the output values ​​in step S606. For example, the execution unit 300 stores the output value in the output register 325, and then the output value is stored at the destination address in the data memory 322. This completes the execution of the interpolation instruction.

[0067] In some examples, some or all of the interpolation instructions may be implemented in dedicated hardware. For example, the execution unit 300 may include specific circuitry for performing comparisons to identify intervals (i.e., step S604), or specific circuitry for interpolation within the identified intervals (i.e., step S605), or both. In some examples, the interpolation instructions may utilize existing hardware, for example, in the FPU.

[0068] Returning to the details in steps S604 and S605, for a LUT 200 having size N (i.e., N entries), if the execution unit 300 can perform K comparisons or interpolations in the execution cycle of the processing unit including the execution unit 300, the maximum number of cycles required to identify an interval and generate an output of a given input vector of width M is equal to P = N / K. As described above, M and N may depend on the platform or implementation. Similarly, K may also depend, more generally, on the platform or implementation of the execution unit 300 and / or the processing unit 4.

[0069] The maximum number of cycles P is, in effect, the worst-case scenario where at least one of the input values ​​in the input vector belongs to the last interval. In examples where the input values ​​do not correspond to that worst-case scenario, the number of cycles required to compute the output is less than P. Naturally, the actual number of cycles required to generate the output is not known in advance.

[0070] For example, in order to enable instructions to be executed deterministically (i.e., in a predetermined time), the processing unit 4 may allow only one instruction / input vector to occupy the processing pipeline at a time. In other words, the execution unit 300 operates in such a way that it takes P cycles to produce M outputs.

[0071] In another example, the execution unit 300 may be configured to store information indicating which inputs in the input vector have been mapped to outputs. For example, the execution unit 300 may be associated with an M-bit register 301 (e.g., including an M-bit register), the bits of register 301 being set by the execution unit when the corresponding input values ​​of the input vector are mapped to output values. Thus, once all bits of register 301 are set, the execution unit 300 outputs a vector of output values. The execution unit 300 then increments the program counter. In this example, the execution of an interpolation instruction occupies only the number of execution cycles actually required to perform the interpolation.

[0072] As explained above, in some examples, the size of LUT200 may have a certain fixed capacity that cannot be changed at runtime, as it may be predetermined. In such examples, the size of LUT200 may, in some cases, be larger than the number of intervals that the user specifies to model the function. In other words, the size N of LUT200 may be larger than the number of points 10² that the user defines to model the function. In such examples, some entries within LUT200 will remain empty.

[0073] Considering this situation, the execution unit 300 may be configured to execute the maximum number of cycles. The maximum number of cycles corresponds to the number of (x,y) pairs stored in the LUT200 divided by K. This ensures that comparisons stop after the last (x,y) pair stored in the LUT200 is reached. For example, a register (e.g., a context register 26 such as the control and status register 26CSR) may store the maximum number of cycles. The value in the register may be set by the programmer (i.e., by another computer program instruction).

[0074] In some examples, it will be understood that input values ​​may be outside the function's domain. In other words, input values ​​may be less than the minimum x value x1 or greater than the maximum x value xn+1 in LUT200. In such cases, the execution unit 300 may be configured to handle out-of-domain input values ​​in one of several ways. For example, input values ​​less than x1 may be treated as equal to x1, and input values ​​greater than xn+1 may be treated as equal to xn+1. In other examples, the execution unit may return zero or NaN for input values ​​less than x1 and / or greater than xn+1. In some examples, the user (i.e., the programmer) may specify how input values ​​outside the function's domain are handled, for example, by selecting one of the above options. This may be achieved by setting a control register associated with the instruction to an appropriate configuration value from which a value representing the option is read. Alternatively, the option may be an operand of the instruction.

[0075] In a typical example, the domain is [0,1]. In other words, x1 is 0 and xn+1 is 1.

[0076] In some examples, each interval 101 is assigned to a thread; that is, the step of interpolating between values ​​that define a particular interval is the role of the corresponding thread. For example, there may be n threads, each corresponding to one of the n intervals 101. In some situations, the intervals 101 are non-uniform; for example, shorter intervals 101 can be used to model regions with steeper gradients in a curve. In such situations, if the input is a vector of uniformly distributed values, some intervals (i.e., wider intervals) will correspond to more input values. In other words, the input values ​​are not uniformly distributed across threads. This can result in different execution times for the threads and may necessitate reordering of output values.

[0077] Within the scope of this disclosure, various modifications can be made to the examples described above. For example, an interpolation instruction can be applied to a single input value rather than a vector containing multiple input values. In another example, the context register can store the maximum number of comparisons rather than the maximum number of cycles. While the tile processor 4 has been used as an example, it should be understood that this is merely one example of a suitable processing unit incorporating the execution unit 300 as defined herein.

[0078] Next, the multi-tile processing unit will be described further. As described above, processor 4 can form part of the multi-tile processing unit. There are many possible different forms of suitable processing units that can take the form of a chip. Graphcore has developed an intelligent processing unit (IPU) described, for example, U.S. Patent Application Nos. US2019 / 0121387A1, US2019 / 0121388A1, US2019 / 0121777A1, and US2020 / 0319861A1, the details of which are incorporated herein by reference. Figure 2B is a very schematic diagram of the IPU. The IPU includes a plurality of tiles 1103 on a silicon die, each tile including a processing unit (e.g., processing unit 4 described above) having local memory. The tiles communicate with each other using time-deterministic exchange. The switching fabric 1101 (sometimes called the exchange or exchange fabric) is connected to each tile by its respective set of output wires and connectable to each tile by its respective set of input wires via a switching circuit controllable by each tile. A synchronization module (not shown) is operable to generate a synchronization signal for switching between the compute phase and the exchange phase. Tiles execute their local programs in the compute phase according to a common clock that may be generated on the die or received by the die. At a predetermined time in the exchange phase, a tile can execute a transmit command from its local program to send a data packet to its output set of connecting wires, the data packet destined for at least one receiving tile but without a destination identifier that identifies that receiving tile. At a predetermined switch time, a receiving tile executes a switch control command from its local program to control its switching circuit, connects its set of input wires to the switching fabric, and receives the data packet at the receive time. The transmit time, when the data packet is scheduled to be sent from the transmitting tile, and the predetermined switch time are controlled by a common clock with respect to the synchronization signal.

[0079] Time-deterministic exchange enables efficient transfer between tiles on the die. Each tile has its own local memory providing data storage and instruction storage. As described herein, the IPU is further connected to external memory, which can transfer data to the IPU for use by the tiles via the fabric chip.

[0080] IPU tiles 1103 may be programmed so that data packets sent from their local programs by SEND instructions either access memory (memory access packets) or have another IPU connected within the cluster or system as their destination. In these cases, the data packets are sent to the switching fabric by the originating tile 1103 but are not picked up by the receiving tile within the IPU. Instead, the switching fabric provides the tile to the appropriate connector C1, C2, etc., for external communication from the IPU. Packets intended for off-chip communication are generated to include information that defines their final off-chip destination but does not define the external port from which they are sent. When the code is compiled for the tile, the packet is sent to an external port using the principle of time-deterministic exchange to identify the external port for the packet. For example, a memory access packet may identify a memory address. Packets intended for another IPU may include an identifier for the other IPU. This information is used by routing logic on the fabric chip to correctly route off-chip packets generated by the IPU.

[0081] Figure 2B shows five exemplary regions of an exemplary IPU chip separated by four boundaries 1105 represented by dashed lines. Note that the dashed lines represent abstract boundaries 1105 of abstract regions on the processor chip, shown for illustrative purposes, and the boundaries 1105 do not necessarily represent physical boundaries on the IPU chip.< / m> < / n>

Claims

1. A first computer program instruction is received to input multiple entries into a lookup table, each entry including an x ​​value representing a point on the curve of the function and a corresponding y value, and consecutive entries of the multiple entries form an interval of the function. The plurality of entries are entered into a lookup table stored in the local memory associated with the execution unit. Upon receiving a second computer program instruction indicating an input value, Based on the lookup table, the interval of the function to which the input value belongs is determined. An execution unit configured to interpolate between the y values ​​of entries forming the aforementioned interval to generate an output value corresponding to the input value.

2. The execution unit according to claim 1, further configured to perform linear interpolation between the y values ​​of the entries forming the interval.

3. The execution unit according to claim 1 or 2, wherein the first computer program instruction includes a field that holds an instruction for a memory address storing the plurality of entries.

4. The aforementioned multiple entries are ordered by their x values, The execution unit according to any one of claims 1 to 3, wherein the execution unit is configured to sequentially scan the x values ​​in the lookup table to determine the interval.

5. The system is configured to sequentially scan the x-values ​​from the minimum x-value to the maximum x-value. The execution unit according to claim 4, configured to determine the interval as an interval including a first x value greater than the input value and the x value immediately preceding the first x value.

6. The system is configured to sequentially scan the x-values ​​from the maximum x-value to the minimum x-value. The execution unit according to claim 4, configured to determine the interval as an interval including a first x value smaller than the input value and the x value immediately following the first x value.

7. The lookup table has a storage capacity greater than the number of entries in the plurality of entries, The execution unit according to any one of claims 4 to 6, wherein the execution unit is configured to terminate the comparison of input values ​​when it reaches the last entry of the plurality of entries stored in the lookup table.

8. The execution unit according to claim 7, wherein the execution unit is configured to perform a predetermined maximum number of comparisons and / or interpolations, the predetermined maximum number corresponding to the number of comparisons and / or interpolations required to reach the last entry of the plurality of entries stored in the lookup table.

9. The execution unit according to any one of claims 1 to 6, further configured to set the size of the lookup table based on an operand of the first computer program instruction or a configuration value associated with the first computer program instruction.

10. The second computer program instruction indicates an input vector containing multiple input values, The execution unit is configured to determine, for each input value of the input vector, the interval of the function to which each input value belongs, based on the lookup table. An execution unit according to any one of claims 1 to 9, configured to interpolate between y values ​​forming each interval to generate output values ​​corresponding to each input value.

11. Includes a mapping register for storing an instruction for the input value of the input vector from which the output value was generated, The execution unit according to claim 10, wherein the execution unit is configured to output an output vector containing each output value when the mapping register indicates that an output value has been generated for all input values.

12. In response to the input value being smaller than the minimum x value, the execution unit is configured to generate an output value corresponding to the y value of the minimum x value, and / or The execution unit according to any one of claims 1 to 11, wherein, in response to the input value being greater than the maximum x value, the execution unit is configured to generate an output value corresponding to the y value of the maximum x value.

13. In response to the input value being smaller than the minimum x value, the execution unit is configured to generate an output value corresponding to zero or NaN. The execution unit according to any one of claims 1 to 11, wherein the execution unit is configured to generate an output value corresponding to zero or NaN in response to the input value being greater than the maximum x value.

14. A processing unit comprising an execution unit according to any one of claims 1 to 13 and local memory.

15. It is configured to run supervisor threads and worker threads, The first computer program instruction is executed as part of the supervisor thread, The processing unit according to claim 14, wherein the second computer program instruction is executed as part of the worker thread.

16. The processing unit according to claim 14 or 15, configured to execute a computer program including the first computer program instruction and the second computer program instruction.

17. Apparatus comprising a plurality of processing units according to any one of claims 14 to 16.

18. The processing unit communicates via an exchange fabric that implements time-deterministic exchange, according to claim 17.

19. A method that is executed in the execution unit, A step of receiving a first computer program instruction to input multiple entries into a lookup table, wherein each entry includes an x ​​value representing a point on the curve of a function and a corresponding y value, and consecutive entries of the multiple entries form an interval of the function. The steps include: inputting the multiple entries into a lookup table stored in local memory associated with the execution unit; The steps include receiving a second computer program instruction indicating an input value, The steps include determining the interval of the function to which the input value belongs based on the lookup table, A method comprising the step of interpolating between the y values ​​of entries forming the interval to generate an output value corresponding to the input value.

20. A first computer program instruction, when executed by an execution unit, causes the execution unit to input a plurality of entries into a lookup table stored in local memory associated with the execution unit, wherein each entry includes an x ​​value representing a point on the curve of a function and a corresponding y value, and the successive entries of the plurality of entries form an interval of the function, and / or A second computer program instruction indicating an input value, which, when executed by the execution unit, results in the execution unit receiving the following: Based on the lookup table, determine the interval of the function to which the input value belongs. A computer program product including a second computer program instruction that interpolates between the y values ​​of entries forming the interval to generate an output value corresponding to the input value.

Images