Initializing the worker thread

Abstract

1. A processing device comprising: at least one execution unit configured to interleave execution of multiple worker threads, each of the worker threads configured to execute a same set of code to operate on different data sets held in an input buffer of a memory of the processing device, and to output result data to an output buffer, wherein instructions are executed to cause a plurality of operand registers, each associated with one of the worker threads, to input one or more variables that enable each worker to determine where in the input buffer its input data set is located, and where result data should be stored.

Description

[Technical Field]

[0001] The present disclosure relates to processing devices including at least one execution unit, and in particular to processing devices including at least one execution unit configured to interleave execution of multiple worker threads. [Background technology]

[0002] A multithreaded processor is a processor capable of executing multiple program threads in parallel with each other. The processor may include some hardware common to the different threads (e.g., common instruction memory, data memory, and / or execution units), but to support multithreading, the processor also includes some dedicated hardware specific to each thread.

[0003] The dedicated hardware includes at least a respective context register file for each of the number of threads that can be executed at once. The term "context," when referring to a multithreaded processor, refers to the program state (e.g., program counter value, status, and current operand values) of each of the threads that execute in parallel with one another. A context register file refers to a respective set of registers for representing the program state of each thread. Registers in a register file differ from typical memory in that register addresses are fixed as bits in an instruction word, whereas memory addresses can be calculated by executing instructions. The registers of a particular context typically include a respective program counter for each thread and a respective set of operand registers for temporarily holding data acted upon and output by each thread during computations performed by that thread. Each context may also have a respective status register for storing the status of each thread (e.g., whether it is paused or running). Thus, each currently executing thread has its own separate program counter and, optionally, operand registers and one or more status registers.

[0004] One possible form of multithreading is parallelism, whereby multiple execution pipelines are provided in addition to multiple contexts (i.e., a separate execution pipeline for each stream of instructions to be executed in parallel). Another form of multithreaded processor employs concurrency over parallelism, whereby threads share a common execution pipeline (or at least a common portion of the pipeline), and different threads are interleaved through this same shared execution pipeline. Nevertheless, the performance of a multithreaded processor may be improved over concurrency or no concurrency due to increased opportunities to hide pipeline latencies. This approach also does not require as much extra hardware dedicated to each thread as a fully parallel processor with multiple execution pipelines, and therefore does not require as much extra silicon.

[0005] In some cases, multithreaded processors may be applied in a data-parallel fashion, where each worker thread executes the same set of code to perform operations on different data sets. In this case, each worker must locate its associated data set in memory to perform the operation. Having each worker individually execute a set of instructions to determine the location of its associated data set can consume a significant number of processor clock cycles, thereby increasing the time it takes to complete the operation. Summary of the Invention [Means for solving the problem]

[0006] According to an embodiment, a processing device is provided that includes at least one execution unit configured to interleave execution of multiple worker threads, each configured to execute the same set of code to operate on different data sets held in an input buffer of a memory of the processing device, and a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by the associated one of the worker threads, wherein the at least one execution unit, in response to a first instruction executed by the at least one execution unit prior to execution of the same set of code, determines, for each of the worker threads, one or more variables that enable the respective worker thread to locate in the input buffer the data set on which the respective worker thread is to operate based on an identifier of the respective worker thread, and causes at least one of the operand registers associated with the respective worker thread to receive the one or more variables determined for the respective worker thread.

[0007] The inventors have recognized that for data parallel operations, a hardware module responsive to a single instruction may be provided to preload worker operand registers with variables that allow it to identify the memory location of the data set to operate on. Because the workers perform the same set of operations, it is sufficient for the hardware module to use the worker identifier to determine the difference in the location of the data set for each worker.

[0008] In some embodiments, for each of the worker threads, one or more variables include an offset into an input buffer where at least one item of the worker thread's data set is stored.

[0009] In some embodiments, each of the worker threads is configured to execute a load instruction to load an item of that dataset from a memory location given by an offset for the worker thread.

[0010] In some embodiments, for each of the worker threads, at least one item in the dataset of the respective worker thread includes a first item to be accessed first in the sequence by the respective worker thread.

[0011] In some embodiments, for each of the worker threads, the respective operations include the same set of operations that are repeatedly performed in a loop, and each of the worker threads is configured to perform each iteration of the loop on a different one of multiple subsets of the dataset.

[0012] In some embodiments, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be executed by the respective worker thread, and each of the worker threads is configured to continue loading data from the input buffer until the number of completed iterations of the loop by the worker thread matches the iteration count of the worker thread.

[0013] In some embodiments, each of the worker threads, when executing the same set of code, is configured to increment a pointer to point to the starting address of the next one of the subsets of the data set between each iteration of the loop.

[0014] In some embodiments, the processing device includes a further register that stores, for each of the worker threads, control information that enables the processing circuitry to determine one or more variables of the respective worker thread.

[0015] In some embodiments, the control information includes an indication, for each of the worker threads, as to whether, in response to execution of the first instruction, one or more variables for the worker thread are input into at least one associated operand register, or whether each of the worker threads individually determines one or more variables that enable it to find the data set in the input buffer.

[0016] In some embodiments, the execution unit is configured to execute a separate instruction for loading control information into a further register before execution of the first instruction.

[0017] In some embodiments, the processing circuitry is configured to receive, for each of the worker threads, a set of input information that enables the processing circuitry to determine one or more variables for the worker thread, the input information including one or more of: a size of an input buffer in which each of the worker thread's data sets is stored; and a size of each subset of the data sets for which the worker thread is configured to perform a single iteration of the same subset of operations that is performed repeatedly in the loop.

[0018] In some embodiments, the processing circuitry is configured to determine, for each of the worker threads, an iteration count based on the size of the input buffer and the number of worker threads.

[0019] In some embodiments, the processing circuitry is configured to determine, for each of the worker threads, a respective offset based at least on the size of each subset of the data set and the number of worker threads.

[0020] In some embodiments, the input information includes control information.

[0021] In some embodiments, the processing circuitry is configured to receive an indication of one of a set of data layout modes for the input buffer and to select, based on the indication of the one of the sets of data layout modes, a set of calculations to be performed to determine the one or more variables.

[0022] In some embodiments, the data layout modes include at least one of an interleaved mode, in which data belonging to different worker threads is interleaved in the input buffer, and a segregated mode, in which, for each worker thread, the worker thread's data set is located in a single contiguous area of ​​the input buffer.

[0023] According to a second aspect, a processing device is provided that includes at least one execution unit configured to interleave execution of multiple worker threads, each configured to execute a same set of code to perform operations on different data sets of input to generate different result sets and store the result sets of the operations in an output buffer of a memory of the processing device, and a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by the associated one of the worker threads, wherein the at least one execution unit, in response to a first instruction executed by the at least one execution unit prior to execution of the same set of code, determines, for each of the worker threads, one or more variables that enable the worker thread to determine a location in the output buffer to which the result set will be written based on an identifier of the worker thread, and causes at least one of the operand registers associated with the respective worker thread to receive the one or more variables determined for the respective worker thread.

[0024] The inventors have recognized that in the case of data parallel operations, a hardware module responsive to a single instruction may be provided to preload worker operand registers with variables that allow identifying memory locations to write the results of the operation. Since the workers perform the same set of operations, it is sufficient for the hardware module to use the worker identifier to determine the difference in the location of each worker's results. The first and second aspects are shown to be closely related alternative solutions to the same technical problem.

[0025] In some embodiments, for each worker thread, one or more variables include a memory address value that enables the worker thread to identify one of the locations in the output buffer to which a portion of the result set will be written.

[0026] In some embodiments, for each of the worker threads, the memory address value is an offset value to an item of the worker thread's input data, and each of the worker threads is configured to identify one of the locations in the output buffer by scaling the memory address to obtain the offset value into the output buffer.

[0027] In some embodiments, each of the worker threads is configured to execute a store instruction to store an item of the result set at a memory location identified by the worker thread's offset value.

[0028] In some embodiments, for each of the worker threads, the operations include the same set of operations performed repeatedly in a loop, and each of the worker threads is configured to perform iterations of the loop to produce a different subset of the result set.

[0029] In some embodiments, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be performed by the worker thread, and each of the worker threads is configured to continue storing its result set until the number of completed iterations of the loop by the worker thread matches the iteration count for that worker thread.

[0030] In some embodiments, each of the worker threads, when executing the same set of code, is configured to increment a pointer between each iteration of the loop to point to the starting address of the next portion of memory where the next subset of the result set should be stored.

[0031] In some embodiments, the processing device includes a further register that stores, for each of the worker threads, control information that enables the processing circuitry to determine one or more variables of the worker thread.

[0032] In some embodiments, the control information includes an indication, for each of the worker threads, as to whether, in response to execution of the first instruction, one or more variables for that worker thread are input into at least one associated operand register, or whether each of the worker threads individually determines the one or more variables.

[0033] In some embodiments, the at least one execution unit is configured to execute a separate instruction for loading control information into a further register prior to execution of the first instruction.

[0034] In some embodiments, the processing circuitry is configured to receive, for each of the worker threads, a set of input information that enables determining one or more variables for the worker thread. The input information includes one or more of: a size of an input buffer containing each of the different input data sets; and a size of each of a plurality of subsets of the result set. One of the worker threads is configured to generate one of the subsets in response to repeatedly performing a single iteration of the same set of operations in a loop.

[0035] In some embodiments, the processing circuitry is configured to determine, for each of the worker threads, an iteration count based on the size of the input buffer and the number of worker threads.

[0036] In some embodiments, the processing circuitry is configured to determine, for each of the worker threads, a memory address value based at least on the size of the subset and the number of worker threads.

[0037] In some embodiments, the input information includes control information.

[0038] In some embodiments, the processing circuitry is configured to receive an indication of one of a set of data layout modes for the output buffer and to select, based on the indication of the one of the set of data layout modes, a set of calculations to be performed to determine the one or more variables.

[0039] In some embodiments, the data layout modes include at least one of an interleaved mode, in which result sets belonging to different worker threads are interleaved in the output buffer, and a separated mode, in which, for each worker thread, the result data set of the worker thread is stored in a single contiguous area of ​​the output buffer.

[0040] In some embodiments, the at least one execution unit is arranged to execute a supervisor thread in addition to the plurality of worker threads, and the supervisor thread is configured to execute a first instruction before launching the worker threads.

[0041] In some embodiments, the first instruction launches a worker thread.

[0042] In some embodiments, the additional register is a register of the supervisor thread.

[0043] In some embodiments, the execution units are configured to execute each of the worker threads in a repeating sequence, the sequence consisting of a number of time slots in which at least one execution unit is operable to interleave the execution of the worker threads.

[0044] In some embodiments, the execution unit is configured to execute a supervisor thread in each of the different time slots prior to launching the worker threads, and to enable the supervisor thread to yield each of the time slots in which it is executing to one of the worker threads in response to execution of the first instruction.

[0045] In some embodiments, the at least one execution unit includes a plurality of execution units, a first execution unit of the plurality of execution units includes a hardware module and is configured to execute the first instruction, and a second execution unit of the plurality of execution units is configured to perform an arithmetic operation on a dataset loaded from memory.

[0046] According to a third aspect, there is provided a method comprising: interleaving execution of a plurality of worker threads, each configured to execute the same set of code to perform operations on different data sets held in an input buffer of a memory of a processing device; and in response to a first instruction executed prior to execution of the same set of code, determining, for each of the worker threads, based on an identifier of the worker thread, one or more variables that enable the worker thread to locate in the input buffer the data set on which the operation is to be performed by the worker thread; and causing at least one operand register associated with the worker thread to be populated with the one or more variables determined for the worker thread.

[0047] In some embodiments, for each of the worker threads, one or more variables include an offset into an input buffer where at least one item of the worker thread's data set is stored.

[0048] In some embodiments, the method includes each of the worker threads executing a load instruction to load an item of the dataset from a memory location given by the offset for the worker thread.

[0049] In some embodiments, for each of the worker threads, at least one item in the dataset of the worker thread includes a first item to be accessed first in a sequence by the worker thread.

[0050] In some embodiments, for each of the worker threads, the operations include the same set of operations performed repeatedly in a loop, and the method includes each of the worker threads performing each iteration of the loop on a different one of the multiple subsets of the data set.

[0051] In some embodiments, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be performed by the worker thread, and the method includes each of the worker threads continuing to load data from the input buffer until the number of iterations of the loop completed by the worker thread matches the iteration count of the worker thread.

[0052] In some embodiments, the method includes incrementing a pointer to point to the starting address of the next one of the subsets of the data set between each iteration of the loop when each of the worker threads executes the same set of code.

[0053] In some embodiments, the method includes storing control information in further registers, the control registers enabling, for each of the worker threads, determination of one or more variables of the respective worker thread.

[0054] In some embodiments, the control information includes an indication, for each of the worker threads, as to whether, in response to execution of the first instruction, one or more variables for that worker thread are input into at least one associated operand register, or whether each of the worker threads individually determines one or more variables that enable it to find its data set in the input buffer.

[0055] In some embodiments, the method includes executing a separate instruction to load control information into a further register before executing the first instruction.

[0056] In some embodiments, the method includes receiving, for each of the worker threads, a set of input information enabling determination of one or more variables of the worker thread, the input information including one or more of: a size of an input buffer in which each of the worker thread's data sets is stored; and a size of each subset of the data sets at which the worker thread is configured to perform a single iteration of the same subset of operations that is performed repeatedly in the loop.

[0057] In some embodiments, the method includes determining, for each of the worker threads, an iteration count based on the size of the input buffer and the number of worker threads.

[0058] In some embodiments, the method includes determining, for each of the worker threads, an offset based at least on the size of each subset of the data set and the number of worker threads.

[0059] In some embodiments, the input information includes control information.

[0060] In some embodiments, the method includes receiving an indication of one of a set of data layout modes for the input buffer and selecting, based on the indication of the one of the sets of data layout modes, a set of calculations to be performed to determine one or more variables.

[0061] In some embodiments, the data layout modes include at least one of an interleaved mode, in which data belonging to different worker threads is interleaved in the input buffer, and a segregated mode, in which, for each worker thread, the data set for that worker thread is located in a single contiguous area of ​​the input buffer.

[0062] According to a fourth aspect, there is provided a computer program comprising computer readable instructions to cause at least one processor to perform the method according to the third aspect or any embodiment thereof.

[0063] According to a fifth aspect, there is provided a non-transitory computer readable medium storing a computer program comprising computer readable instructions to cause at least one processor to perform the method according to the third aspect or any embodiment thereof.

[0064] According to a sixth aspect, there is provided a method comprising: interleaving execution of a plurality of worker threads, each configured to execute a same set of code to perform operations on different sets of input data to generate different result sets and to store the result sets in an output buffer of a memory of a processing device; and in response to a first instruction executed before execution of the same set of code, determining, for each of the worker threads, based on an identifier of the worker thread, one or more variables that enable the worker thread to determine a location in the output buffer to which the result set will be written; and causing at least one operand register associated with the worker thread to be input with the one or more variables determined for the worker thread.

[0065] In some embodiments, for each of the worker threads, one or more variables include a memory address value that enables the respective worker thread to identify one of the locations in the output buffer to which a portion of the result set will be written.

[0066] In some embodiments, for each of the worker threads, the memory address value is an offset value to an item of input data for the worker thread, and the method includes identifying, for each of the worker threads, one of the locations in the output buffer by scaling the memory address to obtain an offset value into the output buffer.

[0067] In some embodiments, the method includes each of the worker threads executing a store instruction to store an item of the result set at a memory location identified by the worker thread's offset value.

[0068] In some embodiments, for each of the worker threads, the operations include the same set of operations performed repeatedly in a loop, and the method includes, for each of the worker threads, performing iterations of the loop to generate a different subset of the result set.

[0069] In some embodiments, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be performed by the worker thread, and the method includes each of the worker threads continuing to store the result set until the number of completed iterations of the loop by the worker thread matches the iteration count for that worker thread.

[0070] In some embodiments, the method includes, when each of the worker threads executes the same set of code, incrementing a pointer between each iteration of the loop to point to the starting address of the next portion of memory where the next subset of the result set should be stored.

[0071] In some embodiments, for each of the worker threads, control information that allows determination of one or more variables of the worker thread is stored in a further register.

[0072] In some embodiments, the control information includes an indication, for each of the worker threads, as to whether, in response to execution of the first instruction, one or more variables for that worker thread are input into at least one associated operand register, or whether each of the worker threads individually determines the one or more variables.

[0073] In some embodiments, the method includes executing a separate instruction to load control information into a further register before executing the first instruction.

[0074] In some embodiments, the method includes receiving, for each of the worker threads, a set of input information enabling determination of one or more variables for the worker thread, the input information including one or more of a size of an input buffer containing each of the different input data sets and a size of each of a plurality of subsets of the result set, and the method generating one of the subsets in response to one of the worker threads repeatedly performing a single iteration of the same set of operations in a loop.

[0075] In some embodiments, the method includes determining, for each of the worker threads, an iteration count based on the size of the input buffer and the number of worker threads.

[0076] In some embodiments, the method includes determining, for each of the worker threads, a memory address value based at least on the size of each of the subsets and the number of worker threads.

[0077] In some embodiments, the input information includes control information.

[0078] In some embodiments, the method includes receiving an indication of one of a set of data layout modes for the output buffer and selecting, based on the indication of the one of the set of data layout modes, a set of calculations to be performed to determine one or more variables.

[0079] In some embodiments, the data layout modes include at least one of an interleaved mode, in which result sets belonging to different worker threads are interleaved in the output buffer, and a separated mode, in which, for each worker thread, the result data set for that worker thread is stored in a single contiguous area of ​​the output buffer.

[0080] In some embodiments, the method includes executing a supervisor thread in addition to the plurality of worker threads, the supervisor thread configured to execute a first instruction before launching the worker threads.

[0081] In some embodiments, the first instruction launches a worker thread.

[0082] In some embodiments, the additional register is a register of the supervisor thread.

[0083] In some embodiments, the method includes repeatedly executing each of the worker threads in a sequence, the sequence consisting of a number of time slots in which execution of the worker threads is interleaved.

[0084] In some embodiments, the method includes executing a supervisor thread in each of the different time slots before launching the worker threads, and in response to executing a first instruction, enabling the supervisor thread to yield each of the time slots in which it is executing to one of the worker threads.

[0085] In some embodiments, the method includes executing a first instruction using a first execution unit of a plurality of execution units including the hardware module, and performing an arithmetic operation on a data set loaded from memory using a second execution unit of the plurality of execution units.

[0086] According to a seventh aspect, there is provided a computer program comprising computer readable instructions to cause at least one processor to perform the method according to the sixth aspect or any embodiment thereof.

[0087] According to an eighth aspect, there is provided a non-transitory computer readable medium storing a computer program comprising computer readable instructions which, when executed by at least one processor, cause the at least one processor to perform a method according to the sixth aspect or any embodiment thereof.

[0088] For a better understanding of the present invention and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings in which: FIG. [Brief explanation of the drawings]

[0089] [Figure 1] 1 is a schematic diagram of a processing device according to an embodiment of the present application; [Figure 2] FIG. 1 is a schematic block diagram of multiple thread contexts. [Figure 3] 1 illustrates a schematic scheme in which time slots are interleaved. [Figure 4] 1 illustrates a schematic representation of a supervisor thread and multiple worker threads executing in multiple interleaved time slots. [Figure 5] FIG. 1 is a diagram of an execution pipeline including a hardware module for inputting variables into the registers of worker threads that allow identification of data to be processed by those worker threads. [Figure 6] FIG. 10 illustrates the division of input buffers in memory among worker threads when an interleaved layout is applied to the data. [Figure 7] FIG. 10 is a diagram of how each of two worker threads increments its read / write pointer when accessing data in a buffer with an interleaved layout of data. [Figure 8A] 1 is a diagram of the division of input buffers in memory among worker threads when a separate layout is applied to the input buffers and the buffer size is a multiple of the number of threads. [Figure 8B]10 is a diagram of the division of input buffers in memory among worker threads when a separate layout is applied to the input buffers and the buffer size is not a multiple of the number of threads. [Figure 9] FIG. 10 is a diagram of how each of two worker threads increments its read / write pointer when accessing data in a buffer with a separate layout of data. [Figure 10] FIG. 10 is a diagram of the control registers for the first instruction. [Figure 11] FIG. 2 is a diagram of a hardware module according to an embodiment of the present application. [Figure 12] 1 illustrates a first method according to an embodiment of the present application. [Figure 13] 2 illustrates a second method according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0090] Embodiments are directed to a processing device including at least one execution unit configured to interleave execution of multiple worker threads, which may take the form of a tile belonging to a multi-tile processing unit, an example of which is described in more detail in U.S. Patent Application No. 16 / 276,834, which is incorporated by reference.

[0091] FIG. 1 illustrates an example of a processing device 4 according to an embodiment of the present disclosure. The processing device 4 may be one of an array of similar processor tiles 4 on the same chip. The processing device 4 includes a multithreaded processing unit 10 in the form of a barrel-threaded processing unit and local memory 11 (i.e., on the same tile in the case of a multi-tile array or on the same chip in the case of a single-processor chip). A barrel-threaded processing unit is a type of multithreaded processing unit in which the pipeline's execution time is divided into a repeating sequence of interleaved time slots (each time slot may be owned by a particular thread), as will be described in more detail shortly. The memory 11 includes an instruction memory 12 and a data memory 22 (which may be implemented in different addressable memory units or different regions of the same addressable memory unit). The instruction memory 12 stores machine code executed by the processing unit 10, and the data memory 22 stores both data operated on by the executed code and data output by the executed code (e.g., as a result of such manipulation).

[0092] The memory 12 stores various different threads of the program, each containing a respective sequence of instructions for performing a particular task or tasks. Note that an instruction referred to herein means an instance of a machine code instruction, i.e., one of the basic instructions of the processor's instruction set, consisting of an opcode and zero or more operands.

[0093] The programs described herein include multiple worker threads and supervisor subprograms that may be structured as one or more supervisor threads. These will be described in more detail shortly. In embodiments, some or all of the worker threads each take the form of a respective "codelet." A codelet is a specific type of thread, which may also be called an "atomic" thread. It has all the input information it needs to execute from the beginning of the thread (startup). That is, it does not receive any input from any other part of the program or memory after startup. Furthermore, other parts of the program do not use any of the thread's outputs (results) until it finishes (terminates). It is guaranteed to terminate unless it encounters an error. Note that some literature also defines codelets as stateless, i.e., unable to inherit information from the first execution if executed a second time; however, this additional definition is not adopted here. Note that not all worker threads need to be codelets (atomic); in embodiments, some or all of the workers may instead communicate with each other.

[0094] Within processing unit 10, multiple different threads from instruction memory 12 may be interleaved through a single execution pipeline 13 (although typically only a subset of all threads stored in instruction memory may be interleaved at any one time throughout the program). Multithreaded processing unit 10 includes multiple context register files 26, each arranged to represent the state (context) of a different one of the concurrently executing threads, a shared execution pipeline 13 common to the concurrently executing threads, and a scheduler 24 for scheduling the concurrent threads for execution through the shared pipeline in an interleaved, preferably round-robin, manner. Processing unit 10 is connected to a shared instruction memory 12 common to the multiple threads, and a shared data memory 22, again common to the multiple threads.

[0095] Execution pipeline 13 includes a fetch stage 14, a decode stage 16, and an execute stage 18 that includes execution units that may perform arithmetic and logical operations, address calculations, load and store operations, and other operations as defined by the instruction set architecture. As will be described, in embodiments, the processing device includes multiple such execution pipelines 13 and execution units 18. However, for simplicity, FIG. 1 shows only one such execution pipeline 13 and execution unit 18. Each of the context register files 26 includes a respective set of registers for representing the program state of the respective thread.

[0096] 2 shows a schematic diagram of one example of registers that make up each context register file 26. Each context register file 26 includes one or more control registers 28, which in embodiments also include at least a program counter (PC) for the respective thread (to track the instruction address the thread is currently executing) and a set of one or more status registers (SRs) that record the current status of the respective thread (whether it is currently executing or whether it has paused, for example, because it encountered an error). Each context register file 26 also includes a respective set of operand registers (OP) 32 for temporarily holding operands of instructions executed by the respective thread, i.e., values ​​that are operated on by the opcodes of the instructions of the respective thread during execution or that result from the operations defined by those opcodes. It will be understood that each context register file 26 may optionally include one or more other types of registers (not shown). It should be noted that although the term "register file" is sometimes used to refer to a group of registers within a common address space, this is not necessarily the case in the present disclosure, and each hardware context 26 (each of the register sets 26 representing each context) may more generally include one or more such register files.

[0097] As described in more detail below, the disclosed arrangement has one worker context register file CX0,...,CX(M-1) and one additional supervisor context register file CXS for each of M number of threads that can be executed in parallel (M=3 in the illustrated example, but this is not limiting). The worker context register file is reserved to store the context of the worker thread, and the supervisor context register file is reserved to store the context of the supervisor thread. Note that in embodiments, the supervisor context is special in that it has a different number of registers than each worker. Each of the worker contexts preferably has the same number of status registers and operand registers as each other. In embodiments, the supervisor context may have fewer operand registers than each worker. Examples of operand registers that a worker context may have but a supervisor does not include are: floating-point registers, accumulation registers, and / or dedicated weight registers (for holding neural network weights). In embodiments, the supervisor may also have a different number of status registers. Furthermore, in an embodiment, the instruction set architecture of processing device 4 may be configured such that the worker threads and one or more supervisor threads execute several different types of instructions, but also share some instruction types.

[0098] Fetch stage 14 is connected to fetch instructions to be executed from instruction memory 12 under the control of scheduler 24. Scheduler 24 is configured to control fetch stage 14 to fetch instructions in turn from each concurrently executing thread of a set of concurrently executing threads in a repeating sequence of time slots, thereby dividing the resources of pipeline 13 into multiple time-interleaved time slots, as will be explained in more detail shortly. For example, the scheduling scheme may be round robin or weighted round robin. Another term for a processor operating in this manner is a barrel-threaded processor.

[0099] In some embodiments, scheduler 24 has access to one of the status registers SR for each thread, which indicates whether the thread is paused, so that in practice scheduler 24 controls fetch stage 14 to fetch instructions for only the currently active one of the threads. In embodiments, preferably, each time slot (and corresponding context register file) is always owned by one thread or another. That is, each slot is always occupied by a thread and always included in scheduler 24's sequence. However, the thread occupying any given slot may happen to be paused at the time, in which case the respective thread's instruction fetch will be passed on when the sequence reaches that slot. Alternatively, for example, in a less preferred alternative implementation, it is not excluded that some slots may be temporarily empty and removed from the scheduled sequence. When referring to the number of time slots that an execution unit is operable to interleave, this refers to the maximum number of slots in which execution can occur in parallel, i.e., the number supported by the unit's hardware.

[0100] The fetch stage 14 has access to a program counter (PC) for each context. For each thread, the fetch stage 14 fetches the next instruction for that thread from the next address in program memory 12, as indicated by the program counter. The program counter increments each execution cycle unless a branch instruction takes place. The fetch stage 14 then passes the fetched instruction to the decode stage 16 to be decoded, which then passes an indication of the decoded instruction, along with the decoded addresses of the operand registers 32 specified in the instruction, to the execution unit 18 to be executed. The execution unit 18 has access to the operand registers 32 and the control register 28, which the execution unit 18 can use when executing instructions based on the decoded register addresses, such as in the case of an arithmetic instruction (e.g., by adding, multiplying, subtracting, or dividing the values ​​of two operand registers and outputting the result to another operand register for the respective thread). Alternatively, if the instruction defines a memory access (load or store), the load / store logic of the execution units 18 follows the instruction and loads a value from the data memory into the respective thread's operand register, or stores a value from the respective thread's operand register into the data memory 22. If the instruction instead defines a branch or status change, the execution units change the value of one of the program counters PC or status registers SR accordingly. Note that while an instruction of one thread is being executed by the execution units 18, an instruction from the thread of the next timeslot in the interleaved sequence may be decoded by the decode stage 16, and / or while an instruction is being decoded by the decode stage 16, an instruction from the thread of the succeeding timeslot may be fetched by the fetch stage 14 (although in general, the scope of the present disclosure is not limited to one instruction per timeslot; for example, in alternative scenarios, a batch of two or more instructions may be issued from a particular thread per timeslot).Interleaving therefore advantageously hides latencies in the pipeline 13 in accordance with known barrel threading techniques.

[0101] An example of an interleaving scheme implemented by scheduler 24 is shown in Figure 3. Here, parallel threads are interleaved according to a round-robin scheme, whereby within each round of the scheme, the round is divided into a sequence of time slots S0, S1, S2, ..., each for executing a respective thread. Typically, each slot is one processor cycle long, and the different slots are uniformly sized, although this is not necessarily the case in all possible embodiments; for example, a weighted round-robin scheme is possible, in which some threads get more cycles per execution round than others. In general, barrel threading can employ either a uniform or weighted round-robin schedule, in which case the weighting can be fixed or adaptive.

[0102] This pattern is then repeated, regardless of the sequence per execution round, with each round containing a respective instance of each time slot. Therefore, it should be noted that a time slot, as referred to herein, refers to a repeating location in the sequence, rather than a specific instance of the time slot in a particular repetition of the sequence. In other words, scheduler 24 assigns pipeline 13 execution cycles to multiple time-interleaved (time-multiplexed) execution channels, each containing a repetition of a respective time slot in a repeating sequence of time slots. In the illustrated embodiment, there are four time slots, but this is for illustrative purposes only; other numbers are possible. For example, in one preferred embodiment, there are actually six time slots.

[0103] Regardless of the number of time slots into which the round robin scheme is divided, in accordance with the present disclosure, processing unit 10 includes one more context register file 26 than the number of time slots, i.e., processing unit 10 supports one more context than the number of interleaved time slots into which it can perform barrel threading.

[0104] By way of example, this is shown in Figure 2. As shown in Figure 3, if there are four time slots S0,...,S3, then there are five context register files, here labeled CX0, CX1, CX2, CX3, and CXS. That is, even though there are only four execution time slots S0,...,S3 in a barrel thread scheme, and therefore only four threads can execute in parallel, this specification discloses adding a fifth context register file CXS that also includes a fifth program counter (PC), a fifth set of operand registers 32, and in embodiments, a fifth set of one or more status registers (SRs). However, as discussed above, in embodiments, it should be noted that the supervisor context may be different from the other contexts CX0,...,CX3, and the supervisor threads may support different instruction sets for operating the execution pipeline 13.

[0105] Each of the first four contexts, CX0,...,CX3, is used to represent the state of each of multiple "worker threads" currently assigned to one of four execution time slots, S0,...,S3, to perform the application-specific computational tasks desired by the programmer (note, again, that this may only be a subset of the total number of worker threads for the program as stored in instruction memory 12). However, the fifth context, CXS, is reserved for the special function of representing the state of a "supervisor thread" (SV), which is responsible for coordinating the execution of the worker threads, at least in the sense of assigning which worker thread W executes in which time slot, S0, S1, S2,..., at any point throughout the program. Optionally, the supervisor thread may have other "oversight" or coordination responsibilities. For example, the supervisor thread may be responsible for enforcing barrier synchronization to guarantee a particular execution order. For example, if one or more second threads depend on data output by one or more first threads running on the same processor module 4, the supervisor may perform barrier synchronization to ensure that none of the second threads start until the first threads finish. And / or to ensure that one or more threads on a processor module 4 do not start until a particular external data source, such as another tile or processor chip, completes the processing necessary to make that data available. The supervisor thread may also be used to perform other functions related to multiple worker threads. For example, the supervisor thread may be responsible for communicating data externally to the processing device 4 (to receive external data acted upon by one or more of the worker threads and / or to send data output by one or more of the worker threads). In general, the supervisor thread may be used to provide any kind of oversight or coordination function desired by a programmer.For example, as another example, the supervisor may oversee transfers between tile local memory 12 and one or more resources of the wider system (external to array 6), such as a storage disk or a network card.

[0106] Of course, four time slots is merely an example, and it should be noted that in general, in other embodiments, if there are up to M time slots 0,...,M-1 per round, then other numbers may exist, such that processing device 4 includes M+1 contexts CX,...,CX(M-1) and CXS, i.e., one for each worker thread that may be interleaved at any one time, plus an extra context for the supervisor. For example, in one exemplary implementation, there are six time slots and seven contexts.

[0107] Referring to FIG. 4, in accordance with the teachings herein, the supervisor thread SV does not have its own time slot in the scheme of interleaved execution time slots. The same is true for workers, since the assignment of slots to worker threads is flexibly defined. More precisely, each time slot has its own dedicated context register file (CX0,...,CXM-1) for storing worker context, which is used by the worker when the slot is assigned to the worker, but not when the slot is assigned to the supervisor. When a particular slot is assigned to a supervisor, that slot uses the supervisor's context register file CVS instead. Note that a supervisor always has access to its own context; workers cannot occupy the supervisor context register file CXS.

[0108] The supervisor thread SV has the ability to execute in any and all of the time slots S0,...,S3 (or more generally, S0,...,SM-1). The scheduler 24 is configured to begin by assigning supervisor threads to all of the time slots when the entire program is started, i.e., the supervisor SV begins executing in all of S0,...,S3. However, the supervisor thread includes a mechanism to temporarily yield each of its executing slots to a respective worker thread (e.g., initially workers W0,...,W3 in the example shown in FIG. 4) at some later point in time (either immediately or after performing one or more supervisor tasks). This is accomplished by the supervisor thread executing a yield instruction, exemplarily referred to herein as "RUN." In an embodiment, this instruction takes two operands: the address of the worker thread in instruction memory 12 and the address of data related to that worker thread in data memory 22. RUN task_addr,data_addr

[0109] Worker threads are portions of code that can be executed in parallel with one another, each representing one or more respective computational tasks to be performed. A data address can specify data to be acted upon by a worker thread. Alternatively, a yield instruction can take only a single operand that specifies the address of a worker thread, and the data address can be included in the code of the worker thread. Alternatively, in another example, the single operand can point to a data structure that specifies the address of a worker thread and data. As mentioned above, in embodiments, at least some of the workers can take the form of codelets, i.e., atomic units of code that can be executed in parallel. Alternatively or additionally, some of the workers need not be codelets, but instead can communicate with each other.

[0110] The yield instruction ("RUN") causes the scheduler 24 to yield the current time slot in which this instruction itself executes to the worker thread specified by an operand. Note that with a yield instruction, it is implicit that the time slot in which this instruction executes is yielded by the instruction (implicit, in the context of machine code instructions, means that no operand is required to specify this; it is implicitly understood from the opcode itself). The time slot that is yielded is therefore the time slot in which the supervisor executes the yield instruction. In other words, the supervisor is executing in the same space as it yields to. The supervisor says "run this code at this location," and from this point onwards the iterating slot is (temporarily) owned by the associated worker thread.

[0111] The supervisor thread SV performs a similar operation in each of the other one or more time slots to yield some or all of that time slot to a different respective one of worker threads W0,...,W3 (selected from a larger set W0,...,Wj in instruction memory 12). The supervisor is suspended when it has done so for the last slot (and will resume where it left off when one of the slots is later reclaimed by worker W).

[0112] Thus, the supervisor thread SV can assign different worker threads, each performing one or more tasks, to different execution time slots of the interleaved execution time slots S0,...,S3. When the supervisor thread determines that it is time to run a worker thread, it uses a yield instruction ("RUN") to assign this worker to the time slot in which the RUN instruction was executed.

[0113] The "RUN" instruction described above is useful for starting worker threads when each worker thread is executing a separate codelet, where each worker thread performs a different set of operations on a data set whose location in memory is specified by the RUN instruction.

[0114] In embodiments of this application example, each worker thread is optionally arranged to execute the same codelet to perform the same set of operations on different data sets that are co-located in an input buffer in memory. This processing by each worker results in each worker producing a different set of results, each of which is co-stored in an output buffer in memory. In this case, the workers are said to operate in a data-parallel manner. In situations where workers are operating in a data-parallel manner, it has been found that improved computational efficiency can be achieved by providing the associated execution unit of the hardware module responsive to a single instruction (referred to herein as a "RUNALL" instruction) to determine one or more locations in the input buffer of data for each worker thread. In response to executing the RUNALL instruction, the execution unit populates the worker thread's operand registers with one or more variables identifying the location of data for the respective worker and launches each of the workers in one of slots S0,...,S3 (or more generally S0,...,S(M-1)). The same variable or variables that identify the location of the data for each worker are also used to enable each worker to determine the location or locations in the output buffer of the data for each worker thread.

[0115] In this specification, an instruction that causes one or more variables to be determined by dedicated hardware is referred to as a RUNALL instruction, but may also be referred to as a "first instruction."

[0116] Reference is now made to Figure 5, which illustrates elements of processing device 4 that are used during execution of the RUNALL instruction. Included within processing device 4 is a main pipeline 500 for execution of particular instructions by the processing device. Main pipeline 500 is responsible for executing memory instructions, such as load and store instructions, and is also responsible for executing the RUNALL instruction. Main pipeline 500 includes an execution unit 510 that takes the form of and operates like execution unit 18 described above with respect to Figure 1.

[0117] The execution unit 510 includes a block of processing logic for the execution unit 510, referred to herein as a hardware module 520. The execution unit 510 also includes an instruction module 530, which includes control circuitry configured to, in response to a RUNALL instruction, cause the hardware module 520 to determine variables (location variables) that enable each worker thread to identify the location of input data and the location for storing results. Having determined these variables, the hardware module 520 stores them in the worker thread's operand registers 32. Each worker is assigned specific data in the input buffer on which it should operate. Each worker repeatedly performs the same set of operations in a loop. The specific amount of data operated on in a single iteration of the loop is referred to herein as an "input atom." An input atom may include multiple data items that are loaded from different locations within the input atom as a result of different load instructions and that are separately operated on by different arithmetic operations. Each iteration of the loop also generates a specific amount of data that is stored in a unit of memory, referred to herein as an "output atom." The output atom may contain multiple data items that are generated as a result of multiple arithmetic operations and stored at different locations within the atom in response to multiple store instructions. Because each iteration of the loop consumes one atom of input data and outputs one atom of output data, the number of atoms in the input buffer and the number of atoms in the output buffer will be the same. However, the size of the input atom need not be the same as the size of the output atom.

[0118] The following discussion refers to determining locations in memory 22 that worker threads access when executing codelets. Accessing an input atom in memory refers to loading the atom's data for use in an operation. Accessing an output atom in memory refers to storing the result of that atom in an output buffer. Unless otherwise specified, teachings related to "buffer" apply to both input and output buffers.

[0119] The location variable determined by hardware module 520 for each worker thread includes an offset into the input buffer in memory 22. The offset into the input buffer in memory 22 specifies the location (from the base address of the input buffer) of the start of the first atom of data in the input buffer that is accessed by the worker thread when executing a codelet.

[0120] The hardware module 520 determines the offset into the input buffer directly. When each worker thread executes its codelet, it determines the offset into the output buffer in memory 22 based on the offset into the input buffer. The offset into the output buffer specifies the location (from the base address of the output buffer) of the start of the first atom of data in the output buffer that is accessed by the worker thread when storing results in the output buffer. The difference between the offset into the input buffer (input offset) and the offset into the output buffer (output offset) depends on the operation performed as part of the codelet. Thus, when a worker thread executes a codelet, it determines the output offset from the input offset.

[0121] The location variable also contains an iteration count value for a particular thread, which specifies the number of atoms of data to be accessed by each thread in memory 22, starting from the first atom of data at the location specified by the offset value. The iteration count is the same for accesses to the input buffer and the output buffer.

[0122] After accessing each atom of data, the worker thread increments its address pointer, which identifies the address within the buffer being accessed. The next atom of data to be accessed is then accessed in memory 22 at the location identified by the incremented address pointer. By providing an offset and the number of atoms of data to be operated on (in the form of an iteration count value), the variables for a particular worker thread provide their associated worker thread with sufficient information to identify each input atom of the data set for that worker thread to process and the location where each output atom produced by that worker thread will be stored.

[0123] Figure 5 shows two of the operand registers 32, denoted $m0 and $m1, for a particular worker thread. These registers belong to a memory register file (MRF) 535. While Figure 5 shows only one exemplary MRF 535 containing one $m0 register and one $m1 register, processing device 4 includes an MRF 535 for each worker thread. The first register ($m0) stores the input offset value for the worker thread, and the second register ($m1) stores the iteration count for the worker thread. These values ​​are used by the worker thread (when it is launched) to determine the location of input atoms in memory 22 and the location in memory 22 where the output atoms of the resulting data will be stored.

[0124] To derive the worker offset and iteration count values, hardware module 520 manipulates the worker ID value, which allows the determination of the portion of the buffer allocated to each worker. The set of manipulations performed by hardware module 520 on the worker ID depends on the layout of the data in the buffer. Here, there are two exemplary layouts: a first, an interleaved layout, where data for different workers is interleaved in the buffer, and a second, a non-interleaved layout, where data for each worker is stored in a contiguous portion of the buffer containing data for only that worker.

[0125] For an interleaved layout, the hardware module 520 determines the input offset for a particular worker as follows: Input offset = Worker ID * Input atom size Equation 1

[0126] The atom size (in Equation 1 and subsequent equations) is the amount of memory space in the buffer used to store data related to a single iteration of the loop. For an input buffer, this amount of memory (input atom size) is the amount used to store the data consumed within a single iteration of the loop when executing a codelet.

[0127] The input offset determined by hardware module 520 is stored in the $m0 register. Each worker thread can use the input offset value in its $m0 register to determine its output offset. Each worker thread determines its output offset from its input offset by applying a scaling factor that depends on the ratio between the amount of input data and the amount of output data. The output offset is given by: Output offset = Input offset * Scaling factor Equation 2

[0128] The scaling factor is determined during codelet compilation and provided as part of the codelet. The scaling factor is available to the worker threads to use as an intermediate operand for the RUNALL instruction to determine the output offset from the input offset.

[0129] Often the scaling factor is a power of 2, for example 0.25, 0.5, 1, 2, 4, etc. Thus, each worker thread is operable to determine the output offset by applying a left or right shift to the input offset.

[0130] Furthermore, for the interleaved layer, the hardware module 520 determines the iteration count as follows:

number

[0131] Equation 3 provides the iteration count for both the output and input buffers.

[0132] Refer to FIG. 6, which illustrates how memory space in buffer 600 (which may be an input buffer or an output buffer) may be distributed among different worker threads when data (i.e., either input data or results) is stored in buffer 600 in an interleaved layout. The buffer's memory addresses increase vertically along the page. Buffer 600 is shown divided into multiple columns, each associated with a different one of the workers. A solid line is shown in each column, representing the respective worker's offset into buffer 600, determined using Equation 1, or, in the case of the output offset, using Equation 2. This worker's offset is the location within buffer 600 of the first atom of data in buffer 600 accessed by that worker. Also shown in each column is a series of dotted lines, representing the starting location in memory 22 of each of the second atoms accessed by the respective worker thread.

[0133] The first worker thread, W0, has a worker ID of 0 and is therefore assigned an offset of zero (according to Equation 1). W0 accesses the first atom of data in buffer 600, as represented by the solid line position in W0's column. The second worker thread has a worker ID of 1 and is therefore assigned an offset of 1 atom. W1 accesses the second atom in buffer 600, as represented by the solid line position in W1's column. Because there are six worker threads, each of the first six atoms of data in buffer 600 is assigned to a different one of the worker threads.

[0134] When buffer 600 is an input buffer into which data items are loaded, each worker thread uses a read pointer value maintained in its MRF 535. When loading its first data item, the read pointer value for a particular worker thread is set to be given by the combination of the base address of the input buffer and the input offset value determined for the respective thread according to Equation 1. After the load operation is completed, the read pointer value is incremented by the stride value to point to the next data item within the same atom. A load operation is then performed to load this next data item before the read pointer value is subsequently incremented by the stride value. Each of these data items is loaded and operated on, with the read pointer value being incremented until the worker reaches the end of the atom. At the end of the atom, the read pointer is incremented by a further stride value to point to the next atom accessed by that worker. This further stride value is greater than the stride value used to increment the read pointer between each data item within the atom. For an interleaved pattern, the further stride value is given by the input atom size multiplied by the number of worker threads minus 1 (e.g., 5 in the example shown in Figure 6) (see equation 4 below). Further data items are loaded starting from the beginning of the next atom of data, which is identified by the updated read pointer value.

[0135] If buffer 600 is an output buffer in which results determined by worker threads are stored, each worker thread uses a write pointer value maintained in its MRF 535. The initial write pointer value for a particular thread is given by the combination of the base address of the output buffer and the output offset value determined for each thread according to Equations 1 and 2. Each worker thread stores its first result data item at the location specified by its initial write pointer value. After completing the store operation, the write pointer value is incremented by the stride value to point to the next location (contiguous with the location of the first result data item) within the same atom. A store operation is then performed to store the next result data item before the write pointer value is again incremented by the stride value. Each result data item is stored, with the write pointer value incremented between each store, until the worker has stored all result data for that iteration of the loop. At the end of storing the result, the write pointer is incremented by a further stride value to point to the next atom accessed by that worker. This additional stride value is larger than the stride value used to increment the write pointer between each data item within each atom. For an interleaved pattern, the additional stride value is the output atom size multiplied by the number of worker threads minus 1 (e.g., 5 in the example shown in Figure 6) (see equation 4 below). The resulting data for the next iteration is then stored starting at the beginning of the next atom of data, which is identified by the updated write pointer value.

[0136] 6, the start of the second atom accessed by thread W0 is displaced from the start of the first atom accessed by thread W0 by the atom size (either the input atom size or the output atom size) multiplied by the number of worker threads (i.e., 6 in this example). However, the start of the second atom accessed by thread W0 is displaced from the end of the first atom accessed by thread W0 (reached by applying a smaller stride value between each data item) by the atom size multiplied by the number of worker threads minus 1 (i.e., 5 in this example). Thus, after each worker thread accesses an atom of data in buffer 600, it increments its pointer value (i.e., write pointer or read pointer) by an amount given by: Interleave stride = atom size * (number of worker threads - 1) Equation 4

[0137] The interleaving stride determined according to Equation 4 is applied from the end of the first atom to determine the start of the second atom accessed by a particular worker thread. The start of the second atom is displaced from the start of the first atom by a number of atoms equal to the number of worker threads.

[0138] Access to atoms of data continues until the end of buffer 600 is reached. The iteration count of each worker thread, determined from Equation 3, allows each worker to determine when it has accessed its last atom of data. Because buffer 600 may not contain a number of atoms that is a multiple of the number of workers, one or more of the workers may have an iteration count that is one greater than the iteration count of one or more of the other workers. In accordance with Equation 3, the iteration count of each worker is determined by dividing the buffer size (represented by atoms) by the number of workers. If the result of this division is not an integer (in which case each worker does not have the same iteration count), the result is rounded down to the nearest integer value. If the worker ID of the respective worker is less than the remainder of dividing the buffer size by the number of workers, an additional value of 1 is added to the rounded result.

[0139] Reference is made to FIG. 7, which shows an example of how atoms for different workers (i.e., input atoms or output atoms) may be arranged in buffer 600 and how the workers adjust their pointers (i.e., read pointers or write pointers) according to a stride value between accesses to each atom of data. In FIG. 7, each numbered element represents an atom. The atoms are shown with addresses increasing horizontally across the page. It will be understood that in practice, buffer 600 will likely contain many more atoms than the number shown in FIG. 7. The number shown in each element represents the ID of the worker accessing the atom of data.

[0140] The arrows are illustrated to show the stride behavior of the first worker (W0) and the second worker (W1), but other workers exhibit similar stride behavior. The stride behavior between locations within each atom is not shown. However, accessing each atom most often involves loading (for input atoms) or storing (for output atoms) multiple data items. After accessing an atom of data in memory, each worker increments its pointer by the stride value determined from Equation 4 (i.e., five atoms in this example) and then accesses the atom of data pointed to by the updated pointer. In the example of FIG. 7, the number of atoms in buffer 600 is such that the number of atoms in buffer 600 is not a multiple of the number of workers, but is one more than a multiple of the number of workers (because worker W0 has one more atom associated with it). As a result, the iteration count of worker W0 is one more than the iteration counts of the remaining workers, and therefore worker W0 performs one more iteration of the codelet's loop.

[0141] For a decoupled layout, the hardware module 520 determines the input offset for a particular worker as follows:

number

[0142] For the interleaved case described above, the output offset is determined by the worker thread applying a scaling factor to the input offset. Thus, the output offset is determined by applying Equation 2, where the input offset is given by Equation 5.

[0143] As in the interleaved case, the iteration count for accessing the input buffer is the same as the iteration count for accessing the output buffer. To determine the iteration count per worker, hardware module 520 determines a value (referred to herein as "max_count") that represents the maximum iteration count of any worker. max_count is given by:

number

[0144] Here, the buffer size is given in terms of the number of atoms in the buffer. If the number of atoms in the buffer is less than or equal to the number of workers, then max_count has a value of 1. Otherwise, at least one worker will have access to more than one atom in the buffer, so max_count has a value of 2 or more.

[0145] Further, to determine the iteration count for each worker, hardware module 520 determines a worker ID value such that workers with ID values ​​less than this determined value have an iteration count given by the max_count value, while workers with ID values ​​equal to or greater than the determined value have a lower iteration count. This determined worker ID value is referred to herein as the "cutoff" and is given by:

number

[0146] where the buffer size is given in terms of the number of atoms in the buffer. If the worker ID is less than this cutoff value, the worker is assigned an iteration count equal to the max_count value. If the worker ID is equal to this cutoff value, the iteration count is set equal to a different value (referred to herein as the "remainder"), which is given by the remainder of the division performed in Equation 7, i.e.: Remainder = Buffer size % max_count Equation 8

[0147] If the worker ID is greater than the cutoff value, the iteration count is set equal to zero.

[0148] Thus, to determine the iteration count for a worker thread, hardware module 520 determines a cutoff value according to Equation 7. Then, for each worker thread, hardware module 520 compares the cutoff value with the worker ID value for the respective worker thread. If the worker ID value is less than the cutoff value, hardware module 520 sets the iteration count equal to max_count. If the worker ID is equal to the cutoff value, hardware module 520 sets the iteration count for that worker equal to the remainder, as determined using Equation 8. If the worker ID value is greater than the cutoff value, hardware module 520 sets the iteration count for that worker equal to zero.

[0149] 8A, which illustrates how memory space for a buffer 800 (which may be an input buffer or an output buffer) may be shared among different worker threads when each worker is assigned a contiguous region of memory. The buffer 800 illustrated in FIG. 8A represents an example in which the buffer 800 is divided evenly among the worker threads, such that each worker thread is assigned an equally sized memory region for storing its associated data (i.e., input data or result data).

[0150] The input offset for each worker thread is determined by hardware module 520 applying Equation 5. The output offset is further determined by each worker thread applying Equation 2 to the input offset determined using Equation 5. As a result, the offset value specifying the location of the first atom for a worker thread will be located either at the beginning of the buffer (for worker W0) or immediately after the memory area allocated to another of the workers. As an example, FIG. 8A shows the offset value for worker W2. This offset value provides an indication of the location of the first atom 810 in buffer 800 accessed by worker thread W2 when executing a codelet. This atom is located adjacent to the last atom 820 in buffer 800 accessed by worker thread W1 when executing a codelet.

[0151] Throughout the execution of a codelet, each worker accesses each atom in its allocated memory space in buffer 800. In the case of an input buffer, as it accesses each input atom, the worker loads each of the data items belonging to one of the atoms. During each load, it applies a stride to its read pointer. When a worker has loaded the last data item belonging to one of its input atoms, the atoms are contiguous, so the same stride can be applied to its read pointer to point to the data item at the beginning of the next atom. Each worker continues to apply a stride to its read pointer until it has loaded the last item of the last atom in its allocated memory area.

[0152] For output buffers, as each output atom is accessed, a worker stores multiple data items in that output atom. During each store, it applies a stride to its write pointer. When a worker stores the last data item belonging to one of its output atoms, the atoms are contiguous, so the same stride can be applied to its write pointer to point to the data item at the beginning of the next atom. Each worker continues to apply a stride to its write pointer until it stores the last data item in the last atom in its allocated memory region.

[0153] Each worker continues to access atoms in buffer 800 until its counter reaches its iteration count. The iteration count for each worker is determined by hardware module 520 applying equations 6-8. In the example of FIG. 8A, the total number of atoms in buffer 800 is a multiple of the number of workers. In this case, hardware module 520 determines a cutoff value to be equal to the number of workers. Because worker IDs are assigned in the range of 0 to n-1 (where n is the number of workers), each worker has a worker ID value below the cutoff value. Therefore, hardware module 520 assigns each of the worker threads the same iteration count, given by the max_count value. As a result, the memory areas of the workers in FIG. 8A are equally sized.

[0154] Refer to FIG. 8B, which illustrates an alternative example of how buffer 850 may be divided among worker threads. In this example, the number of atoms in buffer 850 is not a multiple of the number of worker threads, and as a result, the iteration count of the last worker thread (i.e., the worker thread with the largest offset value, which is worker W5 in this example) is less than the iteration counts of the other worker threads. When determining the iteration count, hardware module 520 determines that the cutoff value is equal to n-1 (n is equal to the number of workers), and therefore equal to the worker ID of the last worker. As a result, hardware module 520 assigns each worker (apart from the last worker) an iteration count value equal to the max_count value determined from Equation 6. Additionally, hardware module 520 assigns the last worker an iteration count value equal to the remainder value determined from Equation 8.

[0155] Reference is made to FIG. 9, which illustrates an example of how atoms of data may be accessed by two different worker threads in portion 900 of either buffer 800 or 850. While this diagram shows atoms of data being accessed by a first worker thread (W1) and a second worker thread (W2), the same could apply to any worker thread assigned contiguous memory regions within buffer 800, 850. Data items for worker W1 are labeled with the letters A, B, or C, and data items for worker W2 are labeled with the letters D, E, or F. The data items may constitute input data loaded from buffer 900 (if buffer 900 is an input buffer) for processing by their associated worker thread. The data items may be result data stored in buffer 900 (if buffer 900 is an output buffer) for processing by their associated worker thread. The data items are shown grouped into atoms, with data items associated with a particular atom labeled with the same letter, A through F, respectively.

[0156] Each worker loads or stores one or more data items within a particular time slot of the barrel thread processor before control is passed to the next worker. Here, an example is described in which each worker accesses only a single data item within each time slot. However, it will be understood that the number of items accessed within a single time slot may vary.

[0157] First, worker W1 executes an instruction provided by hardware module 520 to load or store data item A1 at a memory location identified by its offset value. Next, worker W1 updates its pointer value to a new value by applying a stride to the pointer. The stride value is given by the size of a data item. Before worker W1 loads or stores the next data item A2, worker W2 executes an instruction provided by hardware module 520 to load or store its first data item B1 from a location in memory 22 identified by its offset value. Next, worker W1 updates its pointer value to a new value by applying a stride to the pointer, the stride value being given by the size of a data item. Worker W1 loads or stores its next data item A2 when its assigned time slot is selected in the barrel thread processor. Similarly, worker W2 loads or stores its next data item B2 when its assigned time slot is selected in the barrel thread processor. In this way, the loading or storing of data items by different workers is interleaved in time. Each worker W1, W2 continues to load or store data items, striding between each load / store, until the number of completed iterations of that worker's loop of operations equals that worker's iteration count.

[0158] During the loading of different data items, each worker thread may perform one or more arithmetic operations on the loaded data items and one or more store operations to store the results of those arithmetic operations in memory 22.

[0159] Returning to Figure 5, it is shown how the hardware module 520 receives different variables from registers of the processing device 4, as well as how the hardware module 520 outputs variables (i.e., input offset and iteration count) that are accessible to the worker threads and used by those worker threads to perform load operations from the input buffers 700, 800, 850 and store operations to the output buffers 700, 800, 850 in memory 22.

[0160] As shown, one of the supervisor control and status registers (CSRs) is the RUNALL control register 545. The RUNALL control register 545 stores a specific value used by the hardware module 520 in response to the execution of the RUNALL instruction. See FIG. 10, which illustrates the contents of the RUNALL control register 545. This register 545 stores a value (denoted as TSIZE) representing the total number of atoms in the input buffer containing the worker's data. The register 545 stores a value (denoted as ATOM) representing the size of each atom of data in the input buffer. The size of each atom is given by the square of the value of ATOM. The register 545 stores a value (denoted as PRECALC) that selects which calculations are performed by the hardware module 520 to determine the worker's position variables. The PRECALC can be used to specify at least whether an interleaved mode is applied (in which case the hardware module 520 can determine the position variables according to Equations 1 and 3) or whether a sequential mode is applied (in which case the hardware module 520 can determine the position variables according to Equations 5-8). As shown, some bits of RUNALL control register 545 (bits 23-29 in the illustrated example) may be reserved for adding additional functionality.

[0161] Hardware module 520 can access values ​​from the RUNALL control register and, in response to execution of the RUNALL instruction by execution unit 510, can use these values ​​to determine the location variable. Hardware module 520 accesses the worker ID value for each worker thread located in a register (shown in FIG. 5 as Worker.$wsr.CTXTID_M1) and uses this to also determine the location variable. As can be seen from equations 1-8, to determine the location variable, hardware module 520 also utilizes a value representing the number of worker threads. This number (6 in the illustrated example, but could be another number) is implicit in the RUNALL instruction itself, as it is specific to the architecture of processing device 4.

[0162] In response to execution of the RUNALL instruction, hardware module 520 examines the mode operation (i.e., PRECALC) to determine whether interleaved mode or sequential mode applies. If interleaved mode applies, hardware module 520 uses the worker ID value, atom size, and input buffer size to determine the input offset and iteration count for each worker according to Equations 1 and 3. If interleaved mode applies, hardware module 520 uses the worker ID value, atom size, and input buffer size to determine the offset and iteration count for each worker according to Equations 5-8. In either case, circuitry in processing device 4 causes each input offset value to be written to the $m0 register of the associated worker thread, and each iteration count value to be written to the $m1 register of the associated worker thread.

[0163] The hardware module 520 also receives an address value from a register (denoted as $mVERTEX_BASE) in the MRF 540 associated with the supervisor thread. This address value is a pointer to a structure in memory 22 that contains the base address of the input buffer and the base address of the output buffer. The input buffer base address specifies the starting address value of the input buffer. Similarly, the output buffer base address specifies the starting address value of the output buffer. These base address values ​​are the same for all worker threads. The individual input offset value determined by the hardware module 520 for each worker thread is added to the base address value by each worker thread to provide the address of the first atom of data in the input buffer that is accessed. Similarly, the individual output offset value determined by the hardware module 520 for each worker thread is added to the base address value by each worker thread to provide the address of the first atom of data in the output buffer that is accessed. In response to execution of the RUNALL instruction, the hardware module 520 writes a pointer to the structure containing these base addresses into each worker thread's CSR (denoted as Worker.$VERTEX_BASE). During execution, each worker thread then loads the base addresses of its input and output buffers from the locations identified by the pointers in its CSR.

[0164] As described, the main pipeline includes execution units for performing certain types of operations, including load and store operations, adding data to MRF registers 535 in response to a RUNALL instruction, and performing integer arithmetic operations. Processing device 4 also includes auxiliary pipeline 550, which includes execution units for performing floating-point arithmetic operations on data values ​​loaded into arithmetic register file (ARF) 555. The load operations described above may be performed to load values ​​from an input buffer into registers of ARF 555. These values ​​may then be used in arithmetic operations performed by the execution units of device 4 to generate result values. The result values ​​are then stored back into the output buffer in the manner described.

[0165] In the above description, many operations are described as being performed by worker threads. Unless otherwise described as being performed by a particular worker thread, all operations described as being performed by a worker thread are performed by each worker thread, and each worker thread performs the same operation but operates on its own associated data.

[0166] The embodiments described above are merely examples.

[0167] See FIG. 11 , which illustrates an example of hardware module 520 according to an embodiment. Processing circuit 1220 performs the operations described above as being performed by hardware module 520. Hardware module 520 includes one or more input buffers 1210 for receiving variables (e.g., PRECALC, TSIZE, ATOMSIZE, worker ID) used to calculate position variables (e.g., offset and iteration count). These variables are provided from input buffer 1210 to processing circuit 1220, which uses them to derive position variables in the manner described above. Processing circuit 1220 includes any suitable logic circuitry for this purpose. Hardware module 520 includes one or more output buffers 1230 for storing results of processing performed by processing circuit 1220. These results include position variables that are provided by circuitry in processing device 4 to associated context registers (e.g., $m0 and $m1) of processing device 4.

[0168] 12, which illustrates a method 1200 according to an example embodiment. The method 1200 enables each worker thread to locate its input data in memory.

[0169] At S1210, in response to execution of the first instruction, for each worker thread, based on the identifier of the respective worker thread, hardware module 520 determines one or more variables that enable the respective worker thread to locate in the input buffer the data set on which the respective worker thread is to operate.

[0170] At S1220, in response to executing the first instruction, the processing device circuitry causes at least one of the operand registers associated with the respective worker threads to input the one or more variables determined for the respective worker thread, each of the operand registers being associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by its associated one of the worker threads.

[0171] In S1230, at least one execution unit interleaves the execution of multiple worker threads.

[0172] Reference is now made to Figure 13, which illustrates a method 1300 according to an example embodiment. The method 1300 enables each of the worker threads to identify a location in memory where to store its resulting data.

[0173] At S1310, in response to execution of the first instruction, for each worker thread, based on the identifier of the respective worker thread, hardware module 520 determines one or more variables that enable the respective worker thread to determine the location in the output buffer to which the respective result set will be written.

[0174] At S1320, in response to executing the first instruction, hardware module 520 causes at least one operand register associated with each worker thread to input the one or more variables determined for the respective worker thread, each of the operand registers being associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by its associated one of the worker threads.

[0175] In S1330, at least one execution unit interleaves the execution of multiple worker threads.

[0176] It will be understood that the embodiments are described by way of example only.

Claims

1. at least one execution unit configured to interleave execution of multiple worker threads, each worker thread configured to execute the same set of code to operate on different data sets held in an input buffer in the memory of the processing device; a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by the associated one of the worker threads; Including, The at least one execution unit, in response to a first instruction executed by the at least one execution unit prior to execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to find in the input buffer a data set to be operated on by the worker thread; inputting the one or more variables determined for the worker thread into at least one of the operand registers associated with the worker thread; 1. A processing device comprising: a hardware module including processing circuitry configured to perform

2. 2. The processing device of claim 1, wherein for each of the worker threads, the one or more variables include an offset into the input buffer at which at least one item of the worker thread's data set is stored.

3. 3. The processing device of claim 2, wherein each of the worker threads is configured to execute a load instruction to load an item of its dataset from a memory location given by the offset for that worker thread.

4. 4. A processing device according to claim 2 or 3, wherein for each of the worker threads, at least one item of the dataset of the worker thread comprises a first item to be accessed first in a sequence by the worker thread.

5. 5. The processing device of claim 1, wherein for each of the worker threads, the operations comprise the same set of operations performed repeatedly in a loop, and each of the worker threads is configured to perform each iteration of the loop on a different one of a plurality of subsets of a dataset.

6. 6. The processing device of claim 5, wherein, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be executed by the worker thread, and each of the worker threads is configured to continue loading data from the input buffer until the number of completed iterations of the loop by the worker thread matches the iteration count for the worker thread.

7. 7. The processing device of claim 5, wherein each of the worker threads is configured, when executing the same set of code, to increment a pointer to point to the starting address of a next one of the subsets of the data set between each iteration of the loop.

8. 8. A processing device according to any preceding claim, comprising, for each of the worker threads, a further register storing control information that enables the processing circuitry to determine the one or more variables of the worker thread.

9. The control information is for each of the worker threads, whether or not one or more variables for the worker thread are present in at least one associated operand register in response to execution of the first instruction; or whether each of the worker threads individually determines one or more variables that enable it to find a data set in the input buffer; The processing device of claim 8 , including instructions regarding:

10. 10. A processing device according to claim 8 or 9, wherein the execution unit is configured to execute a separate instruction for loading the control information into a further register before execution of the first instruction.

11. The processing circuitry is configured to receive, for each of the worker threads, a set of input information that enables the processing circuitry to determine one or more variables of the worker thread, the input information comprising: the size of the input buffer in which each of the data sets of the worker threads is stored; and the size of each subset of the dataset, wherein the worker threads are configured to perform a single iteration of the same subset of operations repeated in a loop; A processing device according to any one of claims 1 to 10, comprising one or more of:

12. 12. The processing device of claim 11 when dependent on claim 6, wherein the processing circuitry is configured to determine the iteration count for each of the worker threads based on a size of the input buffer and the number of worker threads.

13. The processing circuitry, for each of the worker threads: the size of the subset of each of the data sets; and the number of worker threads; 13. A processing device according to claim 11 or 12 when dependent on claim 2, configured to determine the offset based at least on:

14. A processing device according to any one of claims 11 to 13 when dependent on claim 8, wherein the input information includes the control information.

15. 15. The processing device of claim 1, wherein the processing circuitry is configured to receive an indication of one of a set of data layout modes for the input buffer and to select a set of calculations to be performed to determine one or more variables based on the indication of the one of the set of data layout modes.

16. The data layout mode is: an interleaved mode in which data belonging to different worker threads are interleaved in the input buffer; and a separation mode in which, for each worker thread, the data set for that worker thread is located in a single contiguous area of ​​the input buffer; The processing device of claim 15 , comprising at least one of:

17. at least one execution unit configured to interleave execution of multiple worker threads, each of which is configured to execute a same set of code to perform operations on different input data sets to generate different result sets and store the result sets of the operations in an output buffer in the memory of the processing device; a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by its associated one of the worker threads; Including, The at least one execution unit, in response to a first instruction executed by the at least one execution unit prior to execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to determine a location in an output buffer to which a result set will be written; inputting the one or more variables determined for the worker thread into at least one of the operand registers associated with the worker thread; 1. A processing device comprising: a hardware module including processing circuitry configured to perform

18. 20. The processing device of claim 17, wherein, for each of the worker threads, the one or more variables include a memory address value that enables the worker thread to identify one of the locations within the output buffer to which a portion of the result set will be written.

19. 20. The processing device of claim 18, wherein for each of the worker threads, the memory address value is an offset value to an item of input data for that worker thread, and each of the worker threads is configured to identify one of the locations within the output buffer by scaling the memory address to obtain an offset value into the output buffer.

20. 20. The processing device of claim 19, wherein each of the worker threads is configured to execute a store instruction to store an item of a result set at a memory location identified by an offset value of the worker thread.

21. 21. The processing device of claim 17, wherein for each of the worker threads, the operations comprise the same set of operations that are repeatedly performed in a loop, and each of the worker threads is configured to perform iterations of the loop to produce a different subset of a result set.

22. 22. The processing device of claim 21, wherein, for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be performed by the worker thread, and wherein each of the worker threads is configured to continue storing result sets until the number of completed iterations of the loop by the worker thread matches the iteration count for the worker thread.

23. 23. A processing device according to claim 21 or 22, wherein each of the worker threads, when executing the same set of code, is configured to increment a pointer between each iteration of the loop to point to the starting address of a next portion of the memory in which a next subset of a result set should be stored.

24. 24. A processing device according to any one of claims 17 to 23, comprising, for each of the worker threads, a further register storing control information that enables the processing circuitry to determine the one or more variables of the worker thread.

25. The control information is for each of the worker threads, whether or not one or more variables for that worker thread are input into at least one associated operand register in response to execution of the first instruction; or whether each of the worker threads individually determines one or more variables; 25. The processing device of claim 24, including instructions regarding:

26. 26. A processing device according to claim 24 or 25, wherein the at least one execution unit is configured to execute a separate instruction for loading the control information into a further register before executing the first instruction.

27. the processing circuitry is configured to receive, for each of the worker threads, a set of input information that enables the processing circuitry to determine one or more variables of the worker thread; the input information includes one or more of: a size of an input buffer containing each of the different input data sets; and a size of each of a plurality of subsets of the result set; 27. A processing device according to any one of claims 17 to 26, wherein one of the worker threads is configured to generate one of the subsets in response to repeatedly performing a single iteration of the same set of operations in a loop.

28. 28. The processing device of claim 27, wherein the processing circuitry is configured to determine, for each of the worker threads, the iteration count based on the size of the input buffer and the number of worker threads.

29. The processing circuitry, for each of the worker threads:

29. A processing device according to claim 27 or 28 when dependent on claim 18, configured to determine the memory address value based at least on the size of the subset and the number of worker threads.

30. 29. A processing device according to claim 27 or 28 when dependent on claim 24, wherein the input information comprises the control information.

31. 31. The processing device of claim 17, wherein the processing circuitry is configured to receive an indication of one of a set of data layout modes for the output buffer and to select a set of calculations to be performed to determine one or more variables based on the indication of the one of the set of data layout modes.

32. The data layout mode is:

32. The processing device of claim 31 , comprising at least one of an interleaved mode, in which the result sets belonging to different worker threads are interleaved in the output buffer, and a segregated mode, in which, for each worker thread, the result data set of the worker thread is stored in a single contiguous area of ​​the output buffer.

33. 33. A processing device according to any preceding claim, wherein at least one execution unit is arranged to execute a supervisor thread in addition to the plurality of worker threads, the supervisor thread being configured to execute the first instruction before launching the worker threads.

34. 34. The processing device of claim 33, wherein the first instruction launches the worker thread.

35. 35. A processing device according to claim 33 or 34 when dependent on claim 9 or 25, wherein the further register is a register of the supervisor thread.

36. 36. A processing device according to any preceding claim, wherein the execution units are configured to execute each of the worker threads in a repeated sequence, the sequence consisting of a number of time slots in which at least one execution unit is operable to interleave execution of the worker threads.

37. The execution unit: executing the supervisor thread in each of different time slots prior to initiating the worker thread; In response to execution of the first instruction, enabling the supervisor thread to yield each of the time slots in which it is executing to one of the worker threads; 37. A processing device according to claim 36 when dependent on claim 33, configured to:

38. the at least one execution unit includes a plurality of execution units, a first execution unit of the plurality of execution units including the hardware module and configured to execute the first instruction; 38. A processing device according to any preceding claim, wherein a second execution unit of the plurality of execution units is configured to perform an arithmetic operation on the data set loaded from the memory.

39. interleaving the execution of a plurality of worker threads, each configured to execute the same set of code to operate on a different data set held in an input buffer in the memory of the processing device; In response to a first instruction being executed before execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to find in the input buffer a data set to be operated on by the worker thread; inputting the one or more variables determined for the worker thread into at least one operand register associated with the worker thread; A method comprising:

40. A non-transitory computer-readable medium storing a computer program comprising a set of executable instructions that cause at least one processor to perform a method, the method comprising: interleaving the execution of a plurality of worker threads, each configured to execute the same set of code to operate on a different data set held in an input buffer in the memory of the processing device; In response to a first instruction being executed before execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to find in the input buffer the data set on which the worker thread is to operate; inputting the one or more variables determined for the worker thread into at least one operand register of the worker thread; 1. A non-transitory computer-readable medium comprising:

41. interleaving the execution of multiple worker threads, each configured to execute the same set of code to operate on a different set of input data to generate a different result set, and to store the result set in an output buffer in the memory of the processing device; In response to a first instruction being executed before execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to determine a location in the output buffer to which a result set will be written; inputting one or more variables determined for said worker thread into at least one operand register associated with said worker thread; A method comprising:

42. A non-transitory computer-readable medium storing a computer program comprising a set of executable instructions that cause at least one processor to perform a method, the method comprising: interleaving the execution of multiple worker threads configured to execute the same set of code to operate on different input data sets to generate different result sets and store the result sets in an output buffer in the memory of the processing device; In response to a first instruction being executed before execution of the same set of code, for each of the worker threads: determining, based on an identifier of the worker thread, one or more variables that enable the worker thread to determine a location in an output buffer to which a result set will be written; inputting one or more variables determined for said worker thread into at least one operand register associated with said worker thread; 1. A non-transitory computer-readable medium comprising:

Images