Execution unit, processing unit, and method for generating random samples

Abstract

The execution unit has access to local memory that stores a lookup table having multiple entries, each entry containing an x ​​value and a corresponding y value representing a point on the curve of a cumulative distribution function CDF, and consecutive entries of multiple entries form an interval of the CDF. The execution unit is configured to receive one or more computer program instructions and, in response, generate random numbers using random number generation hardware associated with the execution unit, determine the interval of the CDF to which the generated random numbers belong based on the lookup table, and generate a random sample of the CDF by interpolating between the y values ​​of entries that form the interval based on the generated random numbers.

Description

Technical Field

[0001] The present disclosure relates to an execution unit, a processing device, and a method for generating random samples.

Background Art

[0002] Neural networks are used in the fields of machine learning and artificial intelligence. A neural network includes an arrangement of a set of nodes interconnected by links and interacting with each other. The principle of neural network computation is inspired by information on how electrical stimuli transmit information in the human brain. For this reason, nodes are generally called neurons. When considering a neural network as a computational graph, nodes may also be called vertices, and the links between nodes may be called edges. The network can obtain input data, and specific nodes perform operations on the data. The results of these operations are passed to other nodes. The output of each node is called its activation or node value. Each link is associated with a weight. The weights define the connectivity between the nodes of the neural network. Various techniques are known that enable a neural network to learn by changing the values of the weights.

[0003] Figure 1A shows a highly simplified version of one arrangement of nodes in a neural network to illustrate a specific related concept. This type of configuration is commonly used in learning or training and includes an input layer of nodes, one or more hidden layers of nodes, and an output layer of nodes. In practice, each layer has many nodes, and currently there are typically many hidden layers, and current deep learning models contain a large number of layers. Each node in the input layer Ni can generate an activation or node value at its output, which is produced by executing a nonlinear function or activation function on the data provided to that node. The vector of node values ​​from the input layer is scaled by the respective parameter or weight vectors at the input of each node in the hidden layer, and each weight defines the connectivity between that particular node and its connected nodes in the hidden layer. In practice, since networks can have millions or billions of nodes and can be connected multidimensionally, the vectors are often tensors. Node N hThe weights applied to the input are labeled w0···w2. In the example in Figure 1A, each node in the input layer is connected to each node in the hidden layer, at least initially. Each node in the hidden layer can run an activation function on the data provided to it, and similarly generate the output vectors supplied to each node N0 in the output layer N0. This is an example of a fully connected architecture or feedforward network (K. Hornik, M. Stinchcombe, and H. White, "Multilayer Feedforward Networks are Universal Approximators," Neural Networks, vol.2, pp.359-366, 1989), a convolutional architecture (Y. LeCun, K. Kavukcuoglu, and C. Farabet, "Convolutional networks and applications in vision," Proceedings of the 2010 IEEE International Symposium on Circuits and Systems, Paris, France, pp.253-256, 2010), and a transformer model including a self-attention layer (A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, AN Gomez, L. Kaiser, and I. Polosukhin, "Attention is All You Need," arXiv preprint, arXiv:1706.03762). Various other types of architectures have been developed over the years, including [cs.CL], 2017).

[0004] Each node weights its incoming data, for example, by performing a dot product of the node's input activation and its intrinsic weight for each inflow link. An activation function is then executed on the weighted data. The activation function can be, for example, a sigmoid function or a hyperbolic tangent function. See Figure 1B. Recently used activation functions include the rectifier linear unit (ReLU) and the Gaussian error linear unit (GeLU). The network learns by acting on the data input in the input layer, assigning weights to the activations from each node, weighting the data input to each node in the hidden layer, and executing the activation function. Thus, the nodes in the hidden layer act on the weighted data and supply outputs to the nodes in the output layer. The nodes in the output layer can also assign weights to their edges used to generate output node values. While there are different learning approaches, in each case there is forward propagation through the network from left to right in Figure 1A, calculation of the loss (e.g., cross-entropy loss), and backpropagation through the network from right to left in Figure 1A, of the loss gradient. In the next cycle, each node updates its network parameters, taking into account the backpropagated gradient, and generates a modified set of weights. In this way, the network can be trained to perform its desired operation. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] When processing large datasets using neural networks, it is desirable to sample random numbers according to a desired probability distribution function. For example, this can be useful for facilitating Monte Carlo sampling or for adding random noise to weights or activations to reduce overfitting.

[0006] British Patent Publication GB2568660A describes an implementation of a means for generating random numbers based on a probability distribution. The random numbers are generated by hardware pseudorandom number generation and then processed by additional circuitry that provides a desired distribution. The probability distribution is predetermined, i.e., pre-set based on the hardware implementation.

[0007] According to a first aspect of this disclosure, an execution unit has access to local memory storing a lookup table having a plurality of entries, each entry including an x ​​value and a corresponding y value representing a point on the curve of a cumulative distribution function CDF, and consecutive entries of the plurality of entries form an interval of the CDF. The execution unit is Upon receiving one or more computer program instructions, and in response thereto, Random numbers are generated using the random number generation hardware associated with the execution unit. Based on the lookup table, the interval of the CDF to which the generated random number belongs is determined. An execution unit is provided, configured to generate a random sample of the CDF by interpolating between the y values ​​of entries that form the interval based on the generated random numbers.

[0008] The execution unit may be configured to receive a first computer program instruction and generate the random number in response. The execution unit may be configured to receive a second computer program instruction and determine the interval and generate the random sample in response. The execution unit may be configured to receive a single computer program instruction and generate the random number, determine the interval and generate the random sample in response.

[0009] The random number generation hardware may be a pseudo-random number generator (PRNG). The PRNG may be included in the execution unit.

[0010] The execution unit may be configured to perform linear interpolation between the y values ​​of the entries that form the interval.

[0011] The execution unit may be configured to receive a populate instruction and, in response, populate the lookup table with the plurality of entries. The populate instruction may have a field that holds an instruction for a memory address that stores the plurality of entries. The instruction may be an address in data memory that stores the plurality of entries, or a pointer to a register that stores an address in data memory, where the address stored in data memory stores the plurality of entries.

[0012] The multiple entries may be ordered by their x values. The execution unit may be configured to compare the input value with the x values ​​in the lookup table. The execution unit may be configured to sequentially scan the x values ​​in the lookup table and compare the input value with the x values. The initial x value may be the smallest x value, and the execution may be configured to sequentially scan the x values ​​from the smallest x value to the largest x value. The execution unit may be configured to determine the interval as an interval including a first x value greater than the input value and a small x value immediately preceding the first x value. Alternatively, the initial value may be the largest x value, and the execution unit may be configured to sequentially scan the x values ​​from the largest x value to the smallest x value. The execution unit may be configured to determine the interval as an interval including a first x value smaller than the input value and a x value immediately following the first x value.

[0013] The lookup table may have a storage capacity greater than the number of entries in the plurality of entries. The execution unit may be configured to terminate the comparison between input values ​​when it reaches the last entry of the plurality of entries stored in the lookup table. The execution unit may be configured to execute a second computer program instruction by performing a predetermined maximum number of comparisons and / or interpolations, the predetermined maximum number corresponding to the number of comparisons and / or interpolations required to reach the last entry of the plurality of entries stored in the lookup table. The execution unit may be configured to perform K comparisons and / or interpolations per execution cycle. The execution unit may be configured to perform a maximum number of execution cycles, the maximum number of execution cycles corresponding to the maximum number of comparisons and / or interpolations divided by K. The execution unit may be configured to access a comparison register that stores the maximum number of execution cycles or the maximum number of comparisons and / or interpolations. The execution unit may be configured to execute a third (e.g., store) computer program instruction to store the maximum number in the comparison register.

[0014] The execution unit may be configured to set the size of the lookup table based on the operands of the first computer program instruction or configuration values ​​associated with the first computer program instruction.

[0015] The execution unit may be configured to generate a plurality of random numbers in response to one or more computer program instructions. The execution unit may be configured to determine, for each of the plurality of random numbers, the interval of the function to which each random number belongs, based on the lookup table. The execution unit may be configured to generate a random sample corresponding to each random number by interpolating between the y values ​​that form each interval.

[0016] The execution unit may include a mapping register for storing instructions for a plurality of random numbers from which random samples have been generated. When the mapping register indicates that random samples have been generated for all of the plurality of random numbers, the execution unit can output an output vector containing each of the random samples. The mapping register may contain a number of bits equal to the number of the plurality of random numbers.

[0017] The execution unit may be configured to determine that the generated random number lies outside the domain of the function. The domain of the function may be defined by the minimum and maximum x values ​​of the lookup table. In response that the generated random number is less than the minimum x value, the execution unit may generate a random sample corresponding to the y value of the minimum x value. In response that the generated random number is greater than the maximum x value, the execution unit may generate a random sample corresponding to the y value of the maximum x value. In response that the input value lies outside the domain, the execution unit may generate an output value corresponding to zero or NaN.

[0018] According to a second aspect of this disclosure, a processing unit is provided which includes the execution unit described in the first aspect. The processing unit may include local memory for optionally storing lookup tables. The processing unit may also be a tile processor.

[0019] The processing unit may be configured to execute supervisor threads and worker threads. The populate instruction may be executed as part of the supervisor thread. One or more computer program instructions that cause the execution unit to generate the random numbers and / or determine the interval and generate the random samples may be executed as part of the worker thread. The processing unit may be configured to execute a computer program that includes the populate instruction and / or one or more computer program instructions that cause the execution unit to generate the random numbers and / or determine the interval and generate the random samples. The first computer program instruction can form part of the execution preamble of the computer program.

[0020] A third aspect of this disclosure provides a processing apparatus including the processing unit described in the second aspect. The processing apparatus may include a plurality of processing units. At least one, preferably all, of the processing units may include the execution unit described in the first aspect. The processing units can communicate via an exchange fabric that implements time-deterministic exchange.

[0021] According to a fourth aspect of the present disclosure, a method performed by an execution unit, the execution unit having access to a local memory storing a lookup table having a plurality of entries, each entry including an x ​​value and a corresponding y value representing a point on the curve of a cumulative distribution function CDF, and consecutive entries of the plurality of entries forming an interval of the CDF, the method, The steps include receiving one or more computer program instructions and responding to them, The steps include generating random numbers using random number generation hardware associated with the execution unit, The steps include determining the interval of the CDF to which the generated random number belongs based on the lookup table, Interpolating between the y-values of the entries that form the interval based on the generated random number to generate a random sample of the CDF is provided, including the step of.

[0022] Further optional features of the method of the fourth aspect are described above in connection with the first, second, and third aspects and can be combined in any combination.

[0023] According to a fifth aspect, when executed by an execution unit having access to a local memory storing a look-up table having a plurality of entries each including an x-value and a corresponding y-value representing a point on the curve of a cumulative distribution function, the execution unit generates a random number using random number generation hardware associated with the execution unit, determines the interval of the CDF to which the generated random number belongs based on the look-up table, A computer program product is provided that includes one or more computer program instructions for interpolating between the y-values of the entries that form the interval based on the generated random number to generate a random sample of the CDF.

[0024] Further optional features of the method of the fourth aspect and the computer program product of the fifth aspect are described above in connection with the first, second, and third aspects and can be combined in any combination.

[0025] The present disclosure also extends to a computer program product that includes instructions which, when executed, cause an execution unit to execute any of the methods described herein. The present disclosure also extends to a non-transitory computer-readable medium that stores instructions which, when executed, cause an execution unit to execute any of the methods described herein.

[0026] To better understand the present invention and to more clearly show how the present invention is implemented, the following will be described by way of example only with reference to the accompanying drawings.

Brief Description of the Drawings

[0027] [Figure 1A] This is a very simplified schematic diagram of a neural network. [Figure 1B] This is a very simplified schematic diagram of a neuron. [Figure 2A] An example of this disclosure is a schematic block diagram of a processor in which an example is implemented. [Figure 2B] An example of a processor chip in which this disclosure is implemented is shown in a schematic diagram. [Figure 3] This is an illustration of an example function. [Figure 4] This is an example of a lookup table. [Figure 5] This is a schematic block diagram of an exemplary execution unit. [Figure 6] This is a schematic flowchart of an example method executed in the execution unit. [Figure 7] This is a schematic block diagram of another example execution unit. [Figure 8] This is a schematic flowchart of an example method executed in the execution unit.

[0028] In drawings, corresponding reference numerals indicate corresponding components. Those skilled in the art will understand that elements in drawings are shown for conciseness and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in drawings may be exaggerated relative to others to facilitate understanding of various examples. Also, common, well-understood elements that are useful or necessary in commercially viable examples are often omitted to make these various examples easier to understand. [Modes for carrying out the invention]

[0029] In general, the examples of this disclosure provide a means for generating random samples of a cumulative distribution function (CDF) using a LUT stored in local memory associated with an execution unit. The LUT stores values ​​that define intervals of the function. The disclosure provides an example in which an execution unit generates random numbers using a hardware pseudorandom number generator in response to an received instruction. The execution unit then interpolates the output values ​​of the random numbers based on the values ​​in the LUT. The output values ​​form random samples of the CDF. Thus, the execution unit can generate samples of the CDF quickly and efficiently.

[0030] In some examples, this disclosure provides instructions that can be executed by an execution unit, enabling programmers to input (populate) a LUT. These “populate” instructions may be executed, for example, by a supervisor thread executed by the execution unit as part of the execution preamble. Thus, samples can be generated according to a desired user-defined distribution without requiring dedicated hardware to accommodate the distribution.

[0031] Furthermore, these techniques allow programmers to generate values ​​that appropriately approximate the relevant function across a desired range of expected input values. Therefore, users with relevant knowledge of the values ​​most likely to be expected in the application context can configure the LUT accordingly. This thus avoids the need for large LUTs that accommodate a very wide range of input values.

[0032] An example is implemented in a processing unit which can take the form of processor 4, which will be described in more detail with reference to Figure 2A. In some examples, processor 4 can take the form of a tile in a multi-tile processing unit. Examples of such multi-tile processing units are described in more detail in the prior application, U.S. Patent Application US2020 / 0319861A1, which is incorporated by reference.

[0033] Referring to Figure 2A, an example of a processor 4 is shown, including details of the execution unit 18 and context register 26. The illustrated processor 4 includes a weight register file 26W and can therefore be particularly suited to machine learning applications in which machine learning models are trained by tuning the weights of those models. However, the examples of the present invention are not limited to machine learning applications and are more broadly applicable. Furthermore, the described processor 4 is a multithreaded processor capable of executing M threads simultaneously. The processor 4 can support the execution of M worker threads and one supervisor thread, where the worker threads perform arithmetic operations on data to produce results, and the supervisor thread coordinates the worker threads and controls the synchronization and send / receive functions of the processor 4.

[0034] The processor 4 includes instruction buffers 53 for each of the M threads that can run concurrently. The context registers 26 include main register files (MRFs) 26M for each of the M worker contexts and supervisor contexts. The context registers further include auxiliary register files (ARFs) 26A for at least each of the worker contexts. The context registers 26 further include a common weight register file (WRF) 26W that can be accessed and read by all currently running worker threads. The WRF can be associated with the supervisor context in that the supervisor thread is the only thread that can write to the WRF. The context registers 26 may also include groups of control state registers 26CSR for each of the supervisor context and worker contexts. The execution unit 18 includes a main execution unit 18M and an auxiliary execution unit 18A. The main execution unit 18M includes a load / storage unit (LSU) 55 and an integer arithmetic logic unit (IALU) 56. The auxiliary execution unit 18A includes at least a floating-point arithmetic unit (FPU).

[0035] In each of the J interleaved time slots S0...SJ-1, the scheduler controls a fetch stage (323, see Figure 5) to fetch at least one instruction for each thread from the instruction memory 11 into one of the J instruction buffers 53 corresponding to the current time slot. In this example, each time slot is one execution cycle of the processor, but other schemes are not excluded (e.g., weighted round robin). In each execution cycle of the processor 4 (i.e., each cycle of the processor clock that clocks the program counter), the fetch stage 323 fetches a single instruction or a small “instruction bundle” (e.g., a 2-instruction bundle or a 4-instruction bundle), depending on the implementation. Each instruction is then issued via a decode stage (324, see Figure 5) to either the LSU 55 or IALU 56 of the main execution unit 18M, or the FPU of the auxiliary execution unit 18A, depending on whether the instruction is a memory access instruction, an integer arithmetic instruction, or a floating-point arithmetic instruction, according to its opcode. The LSU55 and IALU56 of the main execution unit 18M execute their instructions using registers from the MRF26M, with specific registers in the MRF26M specified by the instruction operands. The FPU of the auxiliary execution unit 18A performs operations using registers in the ARF26A and WRF 26W, with specific registers in the ARF specified by the instruction operands. In the example, the registers in the WRF may be implicitly included in the instruction type (i.e., predetermined for that instruction type). The auxiliary execution unit 18A may also include circuitry in the form of internal logic latches for holding some internal states 57 for use when performing one or more operations of the floating-point arithmetic instruction types.

[0036] In an example of fetching and executing instructions within a bundle, the individual instructions within a given instruction bundle are executed simultaneously in parallel via independent pipelines 18M and 18A (shown in Figure 2A). In an example of executing a bundle of two instructions, the two instructions may be executed simultaneously along their respective auxiliary and main pipelines. In this case, the main pipeline is configured to execute instruction types that use MRF, and the auxiliary pipeline is used to execute instruction types that use ARF. Pairing instructions into appropriate complementary bundles may be handled by the compiler.

[0037] Each worker thread context has instances of the main register file (MRF) 26M and the auxiliary register file (ARF) 26A (i.e., there is one MRF and one ARF for each barrel thread slot). The functions described herein in relation to the MRF or ARF should be understood to operate on a per-context basis. However, there is a single shared weight register file (WRF) shared among threads. Each thread can access only the MRF and ARF of its own context 26. However, all currently running worker threads can access the common WRF. Thus, the WRF provides a common set of weights used by all worker threads. In the example, only the supervisor can write to the WRF, and only workers can read from the WRF.

[0038] The instruction set of processor 4 includes at least one type of load instruction, which, when executed, causes LSU 55 to load data from data memory 22 into each ARF 26A of the thread in which the load instruction was executed. The destination location within the ARF is specified by an operand of the load instruction. Another operand of the load instruction specifies an address register within each MRF 26M, which holds a pointer to the address of the data memory (322, see Figure 5) into which the data is loaded. The instruction set of processor 4 also includes at least one type of store instruction, which, when executed, causes LSU 55 to store data from each ARF of the thread in which the store instruction was executed into data memory 322. The source location of the store within the ARF is specified by an operand of the store instruction. Another operand of the store instruction specifies an address register within the MRF, which holds a pointer to the address in data memory 322 into which the data is stored. Generally, an instruction set may include separate types of load instructions and store instructions, and / or at least one type of load and store instruction that combines load and store operations into a single instruction.

[0039] In response to the opcode of the relevant type of arithmetic instruction, the arithmetic unit (e.g., FPU) within the auxiliary execution unit 18A performs the arithmetic operation specified by the opcode, which includes operating on the values ​​of the specified source registers in each ARF of the thread, and optionally on the values ​​of the source registers in the WRF. The result of the arithmetic operation is also output to the destination register of each ARF of the thread, which is explicitly specified by the destination operand of the arithmetic instruction.

[0040] The instruction set of processor 4 also includes the populate and interpolation instructions described herein.

[0041] The processor 4 may include an exchange interface 51 for exchanging data between the memory 11 and one or more other resources, such as other instances of the processor and / or external devices such as a network interface or network-attached storage (NAS) device. As described above, in the example, the processor 4 can form one of an array of interconnected processor tiles, each tile executing a portion of a broader program. Thus, individual processors 4 (tiles) form part of a broader processor or processing system 6. The tiles 4 can be connected to one another via interconnect subsystems, which are connected to each other via their respective exchange interfaces 51. The tiles 4 may be implemented on the same chip (i.e., die), on different chips, or in combination (i.e., the array may be formed from multiple chips, each containing multiple tiles 4). Thus, the interconnect system and the exchange interface 51 may include, accordingly, an internal (on-chip) interconnection mechanism and / or an external (inter-chip) exchange mechanism.

[0042] Figure 3 shows an exemplary function approximated using the techniques described herein. As shown in Figure 3, the curve 100 of a function f that maps input x values ​​to output y values ​​is represented by a series of n intervals 101-1, 101-2, ..., 101-n. Each interval is defined by two points 102 on the curve 100 that form a segment of the curve 100 and form the boundary of interval 101. Each point is represented by a horizontal and vertical coordinate (i.e., x and y coordinates). For example, interval 101-1 is bounded by points 102-1 and 102-2, interval 101-2 is bounded by points 102-2 and 102-3, and interval 101-n is bounded by points 102-n and 102-n+1. Thus, with the exception of the first point 102-1 and the last point 102-n+1, ​​each point acts as the upper bound of the preceding interval and the lower bound of the subsequent interval.

[0043] To map the input value 103 to the output value 104, we identify the x-coordinates of two points that define the interval to which the input belongs. Then, we extract the corresponding y-values. Next, we can calculate the output value by interpolating between the y-values. In the example shown in Figure 3, points 102-2 and 102-3 are the points that define the boundary of the interval 101-2, which contains the input value 103. Therefore, the y-values ​​of these points are used for interpolation.

[0044] In one example, the interpolation between points is linear. Therefore, the overall interpolation technique can be considered a piecewise linear interpolation technique. In such an example, the interpolation technique can consider only the y values ​​that define the boundaries of the identified intervals. For example, in the case shown in Figure 3, the y values ​​of points 102-2 and 102-3 may be sufficient to perform the interpolation.

[0045] However, in other examples, other interpolation techniques can be used. For example, polynomial interpolation techniques such as cubic interpolation or Hermitian interpolation can be used. In these and other examples, the interpolation technique can consider more of the points 102 on both sides of the interval in order to calculate the interpolated output value that belongs to the interval. For example, in the case shown in Figure 3, the y values ​​of points 102-1 and 102-n can be considered. The interpolation can consider two, three, four, or any other number of points on both sides of the interval 101-2 containing the input value 103. The number of points considered on each side of the interval 101-2 does not have to be the same. For example, this may be the case when the determined interval lies between two points near the beginning or end of the function's domain.

[0046] Therefore, when we describe interpolation between two values ​​in this specification, we generally mean generating an output based on two values ​​and belonging to a range between those values. This does not mean that only two values ​​are used to calculate the output.

[0047] FIG. 4 shows a look-up table (LUT) 200 that stores data for performing the function approximation shown in FIG. 3. As shown, LUT 200 includes a plurality of entries that include x-values and corresponding y-values. x1, y1 in LUT 200 correspond to the x-value and y-value of point 102-1, x2, y2 correspond to the x-value and y-value of point 102-2, and so on. The entries in LUT 200 are ordered by their x-values such that x1 < x2, x2 < x3, …, xn < xn+1.

[0048] Next, a process for creating LUT 200 by execution unit 300 will be described in detail with reference to FIGS. 5 and 6. Execution unit 300 can correspond to execution unit 18 described above in connection with FIG. 2A. Elements of execution unit 18 / 300 that are not directly relevant to the present technique are omitted from FIG. 5 for simplicity and clarity.

[0049] As shown in FIG. 5, execution unit 300 is associated with a scratch memory 310 (also referred to as a “scratch pad” memory). Scratch memory 310 is a local memory that forms part of processing unit 4. Memory 310 may be fast and may be physically located near execution unit 300 or incorporated into the execution unit. In one example, memory 310 is a register associated with execution unit 300. For example, memory 310 may be context register 26 described above.

[0050] In step S601 of Figure 6, the execution unit 300 first receives a first instruction (hereinafter referred to as the "populate" instruction) for inputting the LUT 200. For example, the instruction memory 321 contains the populate instruction, which is then fetched by the fetch stage 323 and decoded by the decode stage 324. Next, the execution unit 300 executes the decoded instruction. The populate instruction does not have to be a specific instruction for inputting the LUT 200; instead, it may be a more general memory storage instruction that can store the LUT 200 in the scratch memory 310 when an appropriate operand or configuration value is provided.

[0051] The populate instruction includes an instruction for the location of entry 102 used to populate LUT200 as an operand. For example, the instruction may include an address in data memory 322, or a pointer to a register (not shown) that stores a memory address in data memory 322 containing point 102. In step S602, when the populate instruction is executed, the execution unit 300 loads point 102 from data memory 322 into LUT200.

[0052] Therefore, a programmer can configure LUT200 with values ​​representing arbitrary functions as part of a computer program executed by the execution unit 300. For example, a supervisor thread can issue a populate instruction. The instruction can form part of the execution preamble, i.e., the initial part of the program that sets values ​​for use in the main part of the program, which will be executed later by worker threads. In such an example, LUT200 can then be shared by multiple worker threads. However, in an alternative example, each worker thread can input and access its own LUT200.

[0053] The specific configuration of LUT200 may depend on the platform or implementation.

[0054] In some examples, the size of the LUT200 (i.e., the number of points that can be stored) may be determined in advance (i.e., during the manufacture or configuration of the processing unit). In such examples, a fixed, predetermined memory area 310 may be allocated to the LUT200 (and therefore to the entries in the LUT200). In such examples, not all of the allocated memory may be filled (i.e., not all of the entries in the LUT200 may be filled). Unfilled entries may be set to zero or NaN, or they may contain values ​​set by previous use of the memory 310.

[0055] In other examples, the size of LUT200 may be set by the programmer, for example, using operands or configuration data associated with the populate instruction. In such examples, the size of LUT200 is set to be equal to the number of entries to be stored.

[0056] The precision of the points may also be predetermined. Examples of point precisions include FP32, FP16, BF16, or FP8, but please understand that these are merely examples.

[0057] Next, in step S603, the execution unit 300 receives a second instruction based on the LUT200 to provide an output value corresponding to the received input value. This instruction may be referred to herein as an interpolation or "interp" instruction.

[0058] Interpolation instructions may include instructions for input values ​​as operands. For example, an instruction may include an address of data memory 322, or a pointer to a register (not shown) that stores the memory address of data memory 322 containing the input values. The input values ​​can take the form of a vector of input values.

[0059] Interpolation instructions may be issued by worker threads. Interpolation instructions can form part of a computer program, such as a computer program that trains a neural network.

[0060] For example, an interpolation command takes the following format: f <n> v <m>interp $aDest, $aSrc

[0061] Here, n represents the number of bits in the input value, and m represents the size of the input vector. Examples of n values ​​include 32 bits for full-precision floating-point values, 16 bits for half-precision floating-point values, and 8 bits for quarter-precision floating-point values. Examples of m values ​​include 2, 4, or 8. The operands $aDest and $aSrc represent the memory location where the instruction output is stored and the memory location where the input is stored, respectively.

[0062] In step S604, for the first input value of the input vector, the interval containing the input value is identified. To identify the interval containing the input value, the execution unit 300 performs a sequential comparison between the input value and the x value in LUT200, starting from the first entry x1. In other words, the execution unit 300 sequentially scans the x values ​​in LUT200. Since the x values ​​in the LUT are ordered by increasing the input value, the comparison ends when it is determined that the first x value is greater than the input value. Next, the interval between the first x value xi, which is greater than the input value, and the preceding x value in LUT200, xi-1, is recognized as the interval containing the input value.

[0063] In some examples, the execution unit 300 can alternatively traverse sequentially from the last entry (xn+1) toward x1, gradually decreasing in x-values. In this example, the comparison ends when it is determined that the first x-value is less than the input value. Next, the interval between the first x-value xi, which is less than the input value, and the subsequent x-value xi+1 in the LUT200 is recognized as the interval containing the input value. This substantially corresponds to a conceptual right-to-left sequential traverse, as opposed to a left-to-right sequential traverse of the LUT200 as described above.

[0064] In step S605, the execution unit 300 calculates the y values ​​corresponding to the x values ​​that form the interval. i-1 , y i The y-values ​​are extracted. Next, the execution unit 300 interpolates between the y-values ​​to generate output values ​​corresponding to the input values. The interpolation may be linear interpolation or any other interpolation technique, including those described above.

[0065] In some situations, the first input value is equal to the x value. In one example, the execution unit 300 outputs the corresponding y value, and interpolation is not required. However, this approach may require an additional comparison not only to whether the input value is greater than (or less than in the alternative example) the x value, but also whether it is equal to the x value, to accommodate relatively rare cases. Therefore, in some situations, the execution unit 300 may instead not check whether the input value is equal to the x value, but instead simply apply the relevant interpolation technique. In such cases, the comparison during the sequential scanning process may be whether the x value is greater than or equal to (or less than or equal to in the alternative example) the x value.

[0066] Steps S604 and S605 may be performed for each subsequent input value in the input vector. In one example, steps S604 and S605 may be repeated sequentially for each input value in the input vector.

[0067] However, in another example, step S604 may first be performed for each value of the input vector. That is, each input value is compared with x1, then with x2, and so on, to identify the interval corresponding to each input value. This approach can minimize the number of times the memory storing the LUT is accessed. The comparison of specific input values ​​stops when an interval is identified. Interpolation S605 is then performed for each input value in relation to each identified interval.

[0068] Once all values ​​in the input vector are mapped to their corresponding output values, the execution unit 300 outputs the output values ​​in step S606. For example, the execution unit 300 stores the output value in the output register 325, and then the output value is stored at the destination address in the data memory 322. This completes the execution of the interpolation instruction.

[0069] In some examples, some or all of the interpolation instructions may be implemented in dedicated hardware. For example, the execution unit 300 may include specific circuitry for performing comparisons to identify intervals (i.e., step S604), or specific circuitry for interpolation within the identified intervals (i.e., step S605), or both. In some examples, the interpolation instructions may utilize existing hardware, for example, in the FPU.

[0070] Returning to the details in steps S604 and S605, for a LUT 200 having size N (i.e., N entries), if the execution unit 300 can perform K comparisons or interpolations in the execution cycle of the processing unit including the execution unit 300, the maximum number of cycles required to identify an interval and generate an output of a given input vector of width M is equal to P = N / K. As described above, M and N may depend on the platform or implementation. Similarly, K may also depend, more generally, on the platform or implementation of the execution unit 300 and / or the processing unit 4.

[0071] The maximum number of cycles P is, in effect, the worst-case scenario where at least one of the input values ​​in the input vector belongs to the last interval. In examples where the input values ​​do not correspond to that worst-case scenario, the number of cycles required to compute the output is less than P. Naturally, the actual number of cycles required to generate the output is not known in advance.

[0072] For example, in order to enable instructions to be executed deterministically (i.e., in a predetermined time), the processing unit 4 may allow only one instruction / input vector to occupy the processing pipeline at a time. In other words, the execution unit 300 operates in such a way that it takes P cycles to produce M outputs.

[0073] In another example, the execution unit 300 may be configured to store information indicating which inputs in the input vector have been mapped to outputs. For example, the execution unit 300 may be associated with an M-bit register 301 (e.g., including an M-bit register), the bits of register 301 being set by the execution unit when the corresponding input values ​​of the input vector are mapped to output values. Thus, once all bits of register 301 are set, the execution unit 300 outputs a vector of output values. The execution unit 300 then increments the program counter. In this example, the execution of an interpolation instruction occupies only the number of execution cycles actually required to perform the interpolation.

[0074] As explained above, in some examples, the size of LUT200 may have a certain fixed capacity that cannot be changed at runtime, as it may be predetermined. In such examples, the size of LUT200 may, in some cases, be larger than the number of intervals that the user specifies to model the function. In other words, the size N of LUT200 may be larger than the number of points 10² that the user defines to model the function. In such examples, some entries within LUT200 will remain empty.

[0075] Considering this situation, the execution unit 300 may be configured to execute the maximum number of cycles. The maximum number of cycles corresponds to the number of (x,y) pairs stored in the LUT200 divided by K. This ensures that comparisons stop after the last (x,y) pair stored in the LUT200 is reached. For example, a register (e.g., a context register 26 such as the control and status register 26CSR) may store the maximum number of cycles. The value in the register may be set by the programmer (i.e., by another computer program instruction).

[0076] In some examples, it will be understood that input values ​​may be outside the function's domain. In other words, input values ​​may be less than the minimum x value x1 or greater than the maximum x value xn+1 in LUT200. In such cases, the execution unit 300 may be configured to handle out-of-domain input values ​​in one of several ways. For example, input values ​​less than x1 may be treated as equal to x1, and input values ​​greater than xn+1 may be treated as equal to xn+1. In other examples, the execution unit may return zero or NaN for input values ​​less than x1 and / or greater than xn+1. In some examples, the user (i.e., the programmer) may specify how input values ​​outside the function's domain are handled, for example, by selecting one of the above options. This may be achieved by setting a control register associated with the instruction to an appropriate configuration value from which a value representing the option is read. Alternatively, the option may be an operand of the instruction.

[0077] In a typical example, the domain is [0,1]. In other words, x1 is 0 and xn+1 is 1.

[0078] In some examples, each interval 101 is assigned to a thread; that is, the step of interpolating between values ​​that define a particular interval is the role of the corresponding thread. For example, there may be n threads, each corresponding to one of the n intervals 101. In some situations, the intervals 101 are non-uniform; for example, shorter intervals 101 can be used to model regions with steeper gradients in a curve. In such situations, if the input is a vector of uniformly distributed values, some intervals (i.e., wider intervals) will correspond to more input values. In other words, the input values ​​are not uniformly distributed across threads. This can result in different execution times for the threads and may necessitate reordering of output values.

[0079] Next, an example of the generation of a random sample of the distribution will be described with reference to Figures 7 and 8.

[0080] Figure 7 shows the components corresponding to those described above in relation to Figure 5, with the reference numbers of the corresponding components incremented by 100. Only the differences are explained in detail below.

[0081] In this example, the function approximated by LUT200 is the cumulative distribution function (CDF). The CDF is a distribution that takes the following form: F X (x) = P(X ≤ x)

[0082] The right-hand side represents the probability that the random variable X takes a value less than or equal to x. LUT200 may be input with appropriate values ​​that define the interval of the CDF, as explained above.

[0083] As shown in Figure 7, the execution unit 400 further includes a pseudo-random number generator (PRNG) 402. The PRNG 402 is a hardware unit that includes circuitry configured to generate pseudo-random numbers on demand. For example, each pseudo-random number can take the form of a 64-bit randomized sequence.

[0084] In step S801, the first instruction is received by the execution unit 400. In response, in step S802, the execution unit 400 generates a random number using the PRNG 402. The PRNG 402 can generate multiple pseudorandom numbers. These numbers may be stored, for example, in registers (not shown) in memory 410.

[0085] Next, the generated numbers form an input vector for the interpolation instruction. Thus, the interpolation instruction is executed by the execution unit 400. This allows the execution unit 400 to determine the interval to which the input random numbers belong (step S803, corresponding to step S604 described above). Next, the execution unit interpolates within the identified interval to generate an output value (step S804, corresponding to step S605 described above). This is repeated for all random numbers in the input vector. Next, the generated numbers are output in step S805. The output numbers are a set of CDF samples.

[0086] In this example, the first instruction is used to generate random numbers, and then the second instruction (interpolation instruction) is used to generate samples from those random numbers. However, there are cases where multiple first instructions are executed to generate random numbers. That is, the instruction to generate random numbers can be executed multiple times to generate an appropriate number of random numbers and form the input vector. For example, each execution of the first instruction may return a single random number, so the first instruction is executed a number of times corresponding to the length of the input vector. In another example, the first instruction may return two random numbers, so the first instruction is executed a number of times corresponding to the length of the input vector divided by 2.

[0087] In other examples, a single instruction may be executed that causes random number generation, interval determination, and interpolation. That is, a single executable instruction may include multiple operations, which include the steps described above.

[0088] Within the scope of this disclosure, various modifications can be made to the examples described above. For example, an interpolation instruction can be applied to a single input value instead of a vector containing multiple input values. In other examples, the context register can store the maximum number of comparisons instead of the maximum number of cycles. Although the tile processor 4 has been used as an example, it should be understood that this is merely one example of a suitable processing unit incorporating the execution unit 300 as defined herein. In some examples, the PRNG may be replaced with suitable hardware for generating true random numbers. The PRNG does not need to be part of the execution unit.

[0089] Next, the multi-tile processing unit will be described further. As described above, processor 4 can form part of the multi-tile processing unit. There are many possible different forms of suitable processing units that can take the form of a chip. Graphcore has developed an intelligent processing unit (IPU) described, for example, U.S. Patent Application Nos. US2019 / 0121387A1, US2019 / 0121388A1, US2019 / 0121777A1, and US2020 / 0319861A1, the details of which are incorporated herein by reference. Figure 2B is a very schematic diagram of the IPU. The IPU includes a plurality of tiles 1103 on a silicon die, each tile including a processing unit (e.g., processing unit 4 described above) having local memory. The tiles communicate with each other using time-deterministic exchange. The switching fabric 1101 (sometimes called the exchange or exchange fabric) is connected to each tile by its respective set of output wires and connectable to each tile by its respective set of input wires via a switching circuit controllable by each tile. A synchronization module (not shown) is operable to generate a synchronization signal for switching between the compute phase and the exchange phase. Tiles execute their local programs in the compute phase according to a common clock that may be generated on the die or received by the die. At a predetermined time in the exchange phase, a tile can execute a transmit command from its local program to send a data packet to its output set of connecting wires, the data packet destined for at least one receiving tile but without a destination identifier that identifies that receiving tile. At a predetermined switch time, a receiving tile executes a switch control command from its local program to control its switching circuit, connects its set of input wires to the switching fabric, and receives the data packet at the receive time. The transmit time, when the data packet is scheduled to be sent from the transmitting tile, and the predetermined switch time are controlled by a common clock with respect to the synchronization signal.

[0090] Time-deterministic exchange enables efficient transfer between tiles on the die. Each tile has its own local memory providing data storage and instruction storage. As described herein, the IPU is further connected to external memory, which can transfer data to the IPU for use by the tiles via the fabric chip.

[0091] IPU tiles 1103 may be programmed so that data packets sent from their local programs by SEND instructions either access memory (memory access packets) or have another IPU connected within the cluster or system as their destination. In these cases, the data packets are sent to the switching fabric by the originating tile 1103 but are not picked up by the receiving tile within the IPU. Instead, the switching fabric provides the tile to the appropriate connector C1, C2, etc., for external communication from the IPU. Packets intended for off-chip communication are generated to include information that defines their final off-chip destination but does not define the external port from which they are sent. When the code is compiled for the tile, the packet is sent to an external port using the principle of time-deterministic exchange to identify the external port for the packet. For example, a memory access packet may identify a memory address. Packets intended for another IPU may include an identifier for the other IPU. This information is used by routing logic on the fabric chip to correctly route off-chip packets generated by the IPU.

[0092] Figure 2B shows five exemplary regions of an exemplary IPU chip separated by four boundaries 1105 represented by dashed lines. Note that the dashed lines represent abstract boundaries 1105 of abstract regions on the processor chip, shown for illustrative purposes, and the boundaries 1105 do not necessarily represent physical boundaries on the IPU chip.< / m> < / n>

Claims

1. An execution unit having access to local memory storing a lookup table having multiple entries, each entry including an x ​​value and a corresponding y value representing a point on the curve of a cumulative distribution function CDF, and consecutive entries of the multiple entries form an interval of the CDF, and the execution unit, Upon receiving one or more computer program instructions, and in response thereto, Random numbers are generated using the random number generation hardware associated with the execution unit. Based on the lookup table, the interval of the CDF to which the generated random number belongs is determined. An execution unit configured to generate a random sample of the CDF by interpolating between the y values ​​of entries that form the interval based on the generated random numbers.

2. The system receives a first computer program instruction and, in response, generates the aforementioned random number. The execution unit according to claim 1, configured to receive a second computer program instruction, determine the interval therein, and generate the random sample therein.

3. The execution unit according to claim 1, wherein the execution unit is configured to receive a single computer program instruction and, in response, generate the random number, determine the interval, and generate the random sample.

4. The execution unit according to any one of claims 1 to 3, wherein the random number generation hardware is a pseudo-random number generator PRNG included in the execution unit.

5. The execution unit according to any one of claims 1 to 4, further configured to perform linear interpolation between the y values ​​of the entries forming the interval.

6. An execution unit according to any one of claims 1 to 5, further configured to receive a populate command and, in response, input the plurality of entries into the lookup table.

7. The aforementioned multiple entries are ordered by their x values, The execution unit according to any one of claims 1 to 6, wherein the execution unit is configured to sequentially scan the x values ​​in the lookup table to determine the interval.

8. The system is configured to sequentially scan the x-values ​​from the minimum x-value to the maximum x-value. The execution unit according to claim 7, configured to determine the interval as an interval including a first x value greater than the input value and the x value immediately preceding the first x value.

9. The system is configured to sequentially scan the x-values ​​from the maximum x-value to the minimum x-value. The execution unit according to claim 7, configured to determine the interval as an interval including a first x value smaller than the input value and the x value immediately following the first x value.

10. The lookup table has a storage capacity greater than the number of entries in the plurality of entries, The execution unit according to any one of claims 7 to 9, wherein the execution unit is configured to terminate the comparison between input values ​​when it reaches the last entry of the plurality of entries stored in the lookup table.

11. The execution unit according to claim 10, wherein the execution unit is configured to perform a predetermined maximum number of comparisons and / or interpolations, the predetermined maximum number corresponding to the number of comparisons and / or interpolations required to reach the last entry of the plurality of entries stored in the lookup table.

12. In response to one or more computer program instructions, a plurality of random numbers are generated, For each of the aforementioned plurality of random numbers, the interval of the function to which each random number belongs is determined based on the lookup table. An execution unit according to any one of claims 1 to 11, configured to interpolate between y values ​​forming each interval to generate a random sample corresponding to each random value.

13. In response to the input value being smaller than the minimum x value, the execution unit is configured to generate an output value corresponding to the y value of the minimum x value, and / or The execution unit according to any one of claims 1 to 12, wherein, in response to the input value being greater than the maximum x value, the execution unit is configured to generate an output value corresponding to the y value of the maximum x value.

14. A processing unit comprising an execution unit according to any one of claims 1 to 13 and a local memory for storing a lookup table.

15. It is configured to run supervisor threads and worker threads, The processing unit according to claim 14, wherein the one or more computer program instructions that cause the execution unit to generate the random numbers and / or determine the interval and generate the random samples are executed as part of the worker thread.

16. The processing unit according to claim 15, as dependent on claim 6, wherein the populate instruction is executed as part of the supervisor thread.

17. The processing unit according to claim 12 or 13, configured to execute a computer program that includes one or more computer program instructions.

18. A processing apparatus comprising a plurality of processing units according to any one of claims 14 to 17.

19. The processing unit communicates via an exchange fabric that implements time-deterministic exchange, according to claim 18.

20. A method performed by an execution unit, the execution unit having access to local memory storing a lookup table having a plurality of entries, each entry including an x ​​value and a corresponding y value representing a point on the curve of a cumulative distribution function CDF, and consecutive entries of the plurality of entries forming an interval of the CDF, the method, The steps include receiving one or more computer program instructions and responding to them, The steps include generating random numbers using random number generation hardware associated with the execution unit, The steps include determining the interval of the CDF to which the generated random number belongs based on the lookup table, A method comprising the step of generating a random sample of the CDF by interpolating between the y values ​​of entries that form the interval based on the generated random numbers.

21. When executed by an execution unit having access to local memory that stores a lookup table having multiple entries, each containing an x-value representing a point on the curve of the cumulative distribution function and a corresponding y-value, the execution unit has access to: Random numbers are generated using the random number generation hardware associated with the execution unit. Based on the lookup table, determine the interval of the CDF to which the generated random number belongs. A computer program product comprising one or more computer program instructions that cause a random sample of the CDF to be generated by interpolating between the y values ​​of entries forming the interval based on the generated random numbers.

Images