Initialization of worker threads

Abstract

1. A processing device comprising: at least one execution unit configured to interleave execution of multiple worker threads, each of the worker threads configured to execute a same set of code to operate on different data sets held in an input buffer of a memory of the processing device, and to output result data to an output buffer, wherein instructions are executed to cause a plurality of operand registers, each associated with one of the worker threads, to input one or more variables that enable each worker to determine where in the input buffer its input data set is located, and where result data should be stored.

Description

Technical Field

[0001] The present disclosure relates to a processing device including at least one execution unit, and particularly to a processing device including at least one execution unit configured to interleave the execution of a plurality of worker threads.

Background Art

[0002] A multi-threaded processor is a processor capable of executing a plurality of program threads in parallel with each other. The processor may include certain hardware common to a plurality of different threads (e.g., a common instruction memory, data memory, and / or execution unit), but in order to support multi-threading, the processor also includes certain dedicated hardware unique to each thread.

[0003] Dedicated hardware includes at least one context register file for each of the number of threads that can run at once. “Context,” in the context of a multithreaded processor, refers to the program state (e.g., program counter value, status, and current operand value) of each thread running concurrently with each other. A context register file refers to a set of registers representing the program state of each thread. Registers in a register file differ from general memory in that their register addresses are fixed as bits in the instruction word, whereas their memory addresses can be calculated by executing instructions. The registers of a particular context generally include a program counter for each thread and a set of operand registers for temporarily holding data affected by and output by each thread during calculations performed by that thread. Each context may also have its own status register for storing the status of each thread (e.g., whether it is paused or running). Thus, each currently running thread has its own distinct program counter and optionally operand registers and one or more status registers.

[0004] One possible form of multithreading is parallelism, where multiple execution pipelines are provided in addition to multiple contexts (i.e., a separate execution pipeline for each stream of instructions to be executed in parallel). Another form of multithreaded processor employs concurrency rather than parallelism, where threads share a common execution pipeline (or at least a common part of the pipeline), and different threads interleave through this same shared execution pipeline. Nevertheless, the performance of a multithreaded processor can be improved compared to one without concurrency or parallelism, as it provides more opportunities to hide pipeline latency. This approach does not require as much dedicated extra hardware for each thread as a fully parallel processor with multiple execution pipelines, and therefore requires less extra silicon.

[0005] In some cases, multithreaded processors can be applied in a data-parallel approach, where each worker thread executes the same set of code to perform operations on different datasets. In this case, each worker needs to locate its associated dataset in memory in order to perform the operation. Having each worker execute its instruction set individually to determine the location of the associated dataset consumes a considerable number of processor clock cycles, which can increase the time it takes to complete the operation. [Overview of the Initiative] [Means for solving the problem]

[0006] According to one embodiment, the processing device includes a hardware module including processing circuitry configured to interleave the execution of a plurality of worker threads, each configured to execute the same set of code to perform operations on different datasets held in an input buffer of the processing device's memory, and a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction to be executed by the associated worker thread. The processing device includes a hardware module including processing circuitry configured to, in response to a first instruction executed by the at least one execution unit before the execution of the same set of code, determine for each of the worker threads, based on the identifier of each worker thread, one or more variables that enable each worker thread to find its dataset in the input buffer to be operated on by each worker thread, and cause at least one of the operand registers associated with each worker thread to input the one or more variables determined for each worker thread.

[0007] The inventors have recognized that, in the case of data parallel operations, a hardware module responding to a single instruction may be provided to preload a variable into a worker operand register that enables it to identify the memory location of the dataset to be operated on. Since the workers perform the same set of operations, it is sufficient for the hardware module to use a worker identifier to determine the differences in the dataset locations for each worker.

[0008] In some embodiments, for each worker thread, one or more variables include an offset to an input buffer in which at least one item of the worker thread's dataset is stored.

[0009] In some embodiments, each worker thread is configured to execute a load instruction to load an item of its dataset from a memory location given by an offset relative to the worker thread.

[0010] In some embodiments, for each worker thread, at least one item in the dataset of each worker thread includes a first item that should be accessed first in the sequence by each worker thread.

[0011] In some embodiments, for each worker thread, each operation includes the same set of operations that are repeatedly performed in a loop, and each worker thread is configured to perform each iteration of the loop on one of several distinct subsets of the dataset.

[0012] In some embodiments, for each worker thread, one or more variables include an iteration count indicating the number of loop iterations to be executed by each worker thread, and each worker thread is configured to continue loading data from the input buffer until the number of completed loop iterations by the worker thread matches the worker thread's iteration count.

[0013] In some embodiments, each worker thread is configured to increment a pointer to the next starting address of a subset of the dataset between each iteration of a loop when executing the same set of code.

[0014] In some embodiments, the processing device includes, for each worker thread, a further register that stores control information enabling the processing circuit to determine one or more variables for each worker thread.

[0015] In some embodiments, the control information includes instructions for each worker thread regarding whether, in response to the execution of a first instruction, one or more variables for the worker thread are entered into the associated operand register, or whether each worker thread individually determines one or more variables that enable it to find a dataset in the input buffer.

[0016] In some embodiments, the execution unit is configured to execute a separate instruction to load control information into further registers before executing the first instruction.

[0017] In some embodiments, the processing circuit is configured to receive, for each worker thread, a set of input information that enables the processing circuit to determine one or more variables of the worker thread, the input information including one or more sizes of the input buffer in which each of the worker thread's datasets is stored, and the sizes of each subset of the dataset, which is configured so that the worker thread performs a single iteration of the same subset of operations that are repeated in a loop.

[0018] In some embodiments, the processing circuit is configured to determine the iteration count for each worker thread based on the size of the input buffer and the number of worker threads.

[0019] In some embodiments, the processing circuit is configured to determine the offset for each worker thread based on at least the size of each subset of the dataset and the number of worker threads.

[0020] In some embodiments, the input information includes control information.

[0021] In some embodiments, the processing circuit is configured to receive instructions regarding one of the data layout mode sets for an input buffer and, based on the instructions for one of the data layout mode sets, select a set of calculations to be performed to determine one or more variables.

[0022] In some embodiments, the data layout mode includes at least one of an interleaved mode in which data belonging to different worker threads is interleaved in the input buffer, and an isolated mode in which, for each worker thread, the worker thread's dataset is located in a single contiguous region of the input buffer.

[0023] According to a second embodiment, the processing device includes a hardware module including processing circuitry configured to interleave the execution of a plurality of worker threads, each configured to execute the same set of code to perform operations on different input datasets to produce different result sets and store the result sets of operations in an output buffer of the processing device's memory; and a plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by that associated worker thread. The processing device includes a hardware module including processing circuitry configured to, in response to a first instruction executed by the at least one execution unit before the execution of the same set of code, for each of the worker threads, determine one or more variables based on the worker thread identifier that enable the worker thread to determine the location in the output buffer where the result set is written, and cause at least one of the operand registers associated with each worker thread to input the one or more variables determined for each worker thread.

[0024] The inventors recognized that in the case of data parallel operations, a hardware module responding to a single instruction can be provided to pre-load a variable into a worker operand register that enables identification of the memory location where the result of the process is written. Since workers execute the same set of operations, the hardware module can use a worker identifier to determine the difference in the location of the result for each worker. The first and second aspects show a close association with alternative solutions to the same technical problem.

[0025] In some embodiments, for each of the worker threads, one or more variables include a memory address value that enables identification of one of the positions within an output buffer into which a portion of the result set is written.

[0026] In some embodiments, for each of the worker threads, the memory address value is an offset value relative to an item of the input data of the worker thread, and each of the worker threads is configured to identify one of the positions within the output buffer by scaling the memory address to obtain the offset value to the output buffer.

[0027] In some embodiments, each of the worker threads is configured to execute a store instruction for storing an item of the result set at a memory location identified by the offset value of the worker thread.

[0028] In some embodiments, for each of the worker threads, the operation includes the same set of operations that are repeatedly executed in a loop, and each of the worker threads is configured to execute iterations of the loop to generate different subsets of the result set.

[0029] In some embodiments, for each worker thread, one or more variables include an iteration count indicating the number of iterations of a loop to be performed by the worker thread, and each worker thread is configured to continue storing its result set until the number of completed iterations of the loop by the worker thread matches the iteration count of that worker thread.

[0030] In some embodiments, when each worker thread executes the same code set, between each iteration of the loop, the pointer is configured to increment to point to the start address of the next portion of memory where the next subset of the result set is to be stored.

[0031] In some embodiments, the processing device includes, for each worker thread, an additional register that stores control information that enables the processing circuitry to determine one or more variables of the worker thread.

[0032] In some embodiments, the control information, for each worker thread, includes an instruction regarding whether one or more variables for that worker thread are input into at least one associated operand register in response to the execution of a first instruction, or whether each worker thread determines one or more variables individually.

[0033] In some embodiments, at least one execution unit is configured to execute a separate instruction to load the control information into an additional register prior to the execution of the first instruction.

[0034] In some embodiments, the processing circuit is configured to receive a set of input information for each worker thread, which enables the determination of one or more variables of the worker thread. The input information includes one or more of the sizes of the input buffers containing each of the different input datasets, and the sizes of each of several subsets of the result set. One of the worker threads is configured to generate one of the subsets in response to a single iteration of the same set of operations being repeated in a loop.

[0035] In some embodiments, the processing circuit is configured to determine the iteration count for each worker thread based on the size of the input buffer and the number of worker threads.

[0036] In some embodiments, the processing circuit is configured to determine a memory address value for each worker thread based on at least the size of the subset and the number of worker threads.

[0037] In some embodiments, the input information includes control information.

[0038] In some embodiments, the processing circuit is configured to receive instructions regarding one of the data layout mode sets for an output buffer and, based on the instructions for one of the data layout mode sets, select a set of calculations to be performed to determine one or more variables.

[0039] In some embodiments, the data layout mode includes at least one of an interleaved mode in which result sets belonging to different worker threads are interleaved in the output buffer, and an isolated mode in which, for each worker thread, the worker thread's result dataset is stored in a single contiguous area of ​​the output buffer.

[0040] In some embodiments, at least one execution unit is configured to run a supervisor thread in addition to a plurality of worker threads, and the supervisor thread is configured to execute a first instruction before the worker threads are started.

[0041] In some embodiments, the first instruction starts a worker thread.

[0042] In some embodiments, the additional registers are registers of the supervisor thread.

[0043] In some embodiments, the execution unit is configured to execute each of the worker threads in an iterative sequence, the sequence consisting of multiple time slots in which at least one execution unit can operate to interleave the execution of the worker threads.

[0044] In some embodiments, the execution unit is configured to execute a supervisor thread in each of the different time slots before the worker threads are started, and to allow the supervisor thread to relinquish each of the time slots in which it is running to one of the worker threads in response to the execution of a first instruction.

[0045] In some embodiments, at least one execution unit comprises a plurality of execution units, the first of which comprises a hardware module and is configured to execute a first instruction, and the second of which is configured to perform arithmetic operations on a dataset loaded from memory.

[0046] A third aspect provides a method for interleaving the execution of a plurality of worker threads, each configured to execute the same set of code to perform operations on different datasets held in an input buffer of the memory of a processing device; for each worker thread, in response to a first instruction executed before the execution of the same set of code, determining, based on the worker thread identifier, one or more variables that enable the worker thread to find the dataset in the input buffer on which the worker thread is to operate; and causing at least one operand register associated with the worker thread to be populated with the one or more variables determined for the worker thread.

[0047] In some embodiments, for each worker thread, one or more variables include an offset to an input buffer in which at least one item of the worker thread's dataset is stored.

[0048] In some embodiments, the method includes each worker thread executing a load instruction to load an item of its dataset from a memory location given by an offset relative to the worker thread.

[0049] In some embodiments, for each worker thread, at least one item in the worker thread's dataset includes a first item that should be accessed first in the sequence by the worker thread.

[0050] In some embodiments, for each worker thread, the operations include the same set of operations that are repeatedly performed in a loop, and the method includes each worker thread performing each iteration of the loop on one of several distinct subsets of its dataset.

[0051] In some embodiments, for each worker thread, one or more variables include an iteration count indicating the number of loop iterations to be performed by the worker thread, and the method includes each worker thread continuing to load data from an input buffer until the number of loop iterations completed by the worker thread matches the worker thread's iteration count.

[0052] In some embodiments, the method includes incrementing a pointer to the next starting address of a subset of the dataset between each iteration of the loop, when each worker thread is executing the same set of code.

[0053] In some embodiments, the method includes storing control information in a further register, which enables the determination of one or more variables for each worker thread.

[0054] In some embodiments, the control information includes instructions for each worker thread regarding whether, in response to the execution of a first instruction, one or more variables for that worker thread are entered into the associated operand register, or whether one or more variables are individually determined to enable each worker thread to find its dataset in the input buffer.

[0055] In some embodiments, the method includes executing a separate instruction to load control information into further registers before executing the first instruction.

[0056] In some embodiments, the method includes receiving a set of input information for each worker thread that enables determining one or more variables of the worker thread, the input information including one or more of the size of an input buffer in which each of the worker thread's datasets is stored, and the size of each subset of the dataset configured such that the worker thread performs a single iteration of the same subset of operations repeated in a loop.

[0057] In some embodiments, the method includes determining an iteration count for each worker thread based on the size of the input buffer and the number of worker threads.

[0058] In some embodiments, the method includes determining an offset for each worker thread based on at least the size of each subset of the dataset and the number of worker threads.

[0059] In some embodiments, the input information includes control information.

[0060] In some embodiments, the method includes receiving instructions for one of the data layout mode sets for an input buffer, and selecting a set of calculations to be performed to determine one or more variables based on the instructions for one of the data layout mode sets.

[0061] In some embodiments, the data layout mode includes at least one of an interleaved mode in which data belonging to different worker threads is interleaved in the input buffer, and an isolated mode in which, for each worker thread, the worker thread's dataset is located in a single contiguous region of the input buffer.

[0062] According to a fourth aspect, a computer program is provided which includes computer-readable instructions causing at least one processor to execute the method according to the third aspect or any embodiment thereof.

[0063] According to the fifth aspect, a non-temporary computer-readable medium is provided for storing a computer program which includes computer-readable instructions causing at least one processor to execute the method according to the third aspect or any embodiment thereof.

[0064] A sixth aspect provides a method for interleaving the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different input datasets, producing different result sets, and storing those result sets in an output buffer in the memory of a processing device; and, in response to a first instruction executed before the execution of the same set of code, determining for each worker thread, based on the worker thread identifier, one or more variables that enable the worker thread to determine the location in the output buffer where the result set is written, and causing at least one operand register associated with the worker thread to be populated with the one or more variables determined for the worker thread.

[0065] In some embodiments, for each worker thread, one or more variables include a memory address value that allows each worker thread to identify one of the locations in the output buffer where a portion of the result set is written.

[0066] In some embodiments, for each worker thread, the memory address value is an offset value to an item of input data for the worker thread, and the method includes identifying one of the locations in the output buffer by scaling the memory address to obtain an offset value to the output buffer for each worker thread.

[0067] In some embodiments, the method includes each worker thread executing a store instruction to store an item of the result set at a memory location identified by the worker thread's offset value.

[0068] In some embodiments, for each worker thread, the operations include the same set of operations that are repeatedly performed in a loop, and the method includes iterating through the loop for each worker thread to produce different subsets of the result set.

[0069] In some embodiments, for each worker thread, one or more variables include an iteration count indicating the number of loop iterations to be performed by the worker thread, and the method includes each worker thread continuing to store the result set until the number of completed loop iterations by the worker thread matches the iteration count of that worker thread.

[0070] In some embodiments, the method includes, when each worker thread executes the same set of code, incrementing a pointer between each iteration of the loop to point to the starting address of the next portion of memory where the next subset of the result set should be stored.

[0071] In some embodiments, for each worker thread, control information that enables the determination of one or more variables of the worker thread is stored in an additional register.

[0072] In some embodiments, the control information includes instructions for each worker thread regarding whether, in response to the execution of a first instruction, one or more variables for that worker thread are entered into the associated operand register, or whether each worker thread individually determines one or more variables.

[0073] In some embodiments, the method includes executing a separate instruction to load control information into further registers before executing the first instruction.

[0074] In some embodiments, the method includes receiving, for each worker thread, a set of input information that enables the determination of one or more variables of the worker thread, the input information including the size of an input buffer containing each of different input datasets, and one or more sizes of each of several subsets of the result set, and the method generates one of the subsets in response to one of the worker threads repeatedly performing a single iteration of the same set of operations in a loop.

[0075] In some embodiments, the method includes determining an iteration count for each worker thread based on the size of the input buffer and the number of worker threads.

[0076] In some embodiments, the method includes determining a memory address value for each worker thread based on at least the size of each subset and the number of worker threads.

[0077] In some embodiments, the input information includes control information.

[0078] In some embodiments, the method includes receiving instructions for one of the data layout mode sets for an output buffer, and selecting a set of calculations to be performed to determine one or more variables based on the instructions for one of the data layout mode sets.

[0079] In some embodiments, the data layout mode includes at least one of an interleaved mode in which result sets belonging to different worker threads are interleaved in the output buffer, and an isolated mode in which, for each worker thread, its worker thread's result dataset is stored in a single contiguous area of ​​the output buffer.

[0080] In some embodiments, the method includes executing a supervisor thread in addition to a plurality of worker threads, the supervisor thread being configured to execute a first instruction before the worker threads are started.

[0081] In some embodiments, the first instruction starts a worker thread.

[0082] In some embodiments, the additional registers are registers of the supervisor thread.

[0083] In some embodiments, the method includes executing each worker thread in a repeating sequence, the sequence consisting of multiple time slots in which the execution of the worker threads is interleaved.

[0084] In some embodiments, the method includes running a supervisor thread in each of different time slots before the worker threads are started, and allowing the supervisor thread to relinquish each of the time slots in which it is running to one of the worker threads in response to the execution of a first instruction.

[0085] In some embodiments, the method includes executing a first instruction using a first execution unit among a plurality of execution units including a hardware module, and performing arithmetic operations on a dataset loaded from memory using a second execution unit among the plurality of execution units.

[0086] According to the seventh aspect, a computer program is provided which includes computer-readable instructions causing at least one processor to execute the method according to the sixth aspect or any embodiment thereof.

[0087] According to the eighth aspect, a non-temporary computer-readable medium is provided for storing a computer program which includes computer-readable instructions that cause at least one processor to execute the method according to the sixth aspect or any embodiment thereof when executed by at least one processor.

[0088] Herein, to better understand the present invention and to illustrate how it can be carried out, the attached figures are provided as an example. [Brief explanation of the drawing]

[0089] [Figure 1] This is a schematic diagram of a processing device according to an embodiment of the present application. [Figure 2] This is a schematic block diagram of multiple thread contexts. [Figure 3] A schematic diagram of the scheme in which time slots are interleaved is shown. [Figure 4] This diagram schematically shows a supervisor thread and multiple worker threads running in multiple interleaved time slots. [Figure 5] This is a diagram of an execution pipeline that includes a hardware module for inputting variables into the worker threads' registers, which allow for the identification of the data processed by those worker threads. [Figure 6] This diagram illustrates the partitioning of in-memory input buffers across worker threads when an interleaved layout is applied to the data. [Figure 7] This diagram illustrates how each of two worker threads increments its read / write pointer when accessing data in a buffer with an interleaved data layout. [Figure 8A] This diagram illustrates the partitioning of an input buffer in memory among worker threads when a segregated layout is applied to the input buffer and the buffer size is a multiple of the number of threads. [Figure 8B]This diagram illustrates the partitioning of an input buffer in memory among worker threads when a segregated layout is applied to the input buffer and the buffer size is not a multiple of the number of threads. [Figure 9] This diagram illustrates how each of two worker threads increments its read / write pointer when accessing data in a buffer with a data separation layout. [Figure 10] This is a diagram of the control registers for the first instruction. [Figure 11] This is a diagram of a hardware module according to an embodiment of the present application. [Figure 12] A first method according to an embodiment of this application is shown. [Figure 13] A second method according to an embodiment of this application is shown. [Modes for carrying out the invention]

[0090] The embodiments relate to a processing device comprising at least one execution unit configured to interleave the execution of multiple worker threads. This processing device may take the form of a tile belonging to a multi-tile processing unit. An example of such a multi-tile processing unit is described in detail in U.S. Patent Application No. 16 / 276834, which is incorporated by reference.

[0091] Figure 1 shows an example of a processing device 4 according to an embodiment of the present disclosure. The processing device 4 may be one of an array of similar processor tiles 4 on the same chip. The processing device 4 includes a multithreaded processing unit 10 in the form of a barrel-threaded processing unit and local memory 11 (i.e., on the same tile in the case of a multi-tile array or on the same chip in the case of a single processor chip). A barrel-threaded processing unit is one type of multithreaded processing unit in which the execution time of the pipeline is divided into a repeating sequence of interleaved time slots (each time slot may be owned by a particular thread). This will be described in more detail shortly. The memory 11 includes an instruction memory 12 and a data memory 22 (these may be implemented in different addressable memory units or in different areas of the same addressable memory unit). The instruction memory 12 stores machine code executed by the processing unit 10, and the data memory 22 stores both data manipulated by the executed code and data output by the executed code (e.g., as a result of such operations).

[0092] Memory 12 stores various different threads of the program. Each thread contains its own sequence of instructions for performing a specific one or more tasks. Note that the instructions referred to herein mean machine code instructions, i.e., instances of one basic instruction of the processor's instruction set, each consisting of one opcode and zero or more operands.

[0093] The programs described herein include a number of worker threads and a supervisor subprogram which may be constructed as one or more supervisor threads. These will be described in more detail shortly. In embodiments, each of some or all of the worker threads takes the form of its own “codelet.” A codelet is a specific type of thread, which may also be called an “atomic” thread. It has all the input information it needs to execute from the beginning of the thread (from startup). That is, it does not receive any input from any other part of the program or from memory after startup. Furthermore, no other part of the program uses any output (results) of the thread until it finishes (terminates). Unless it encounters an error, it is guaranteed to terminate. Note that some literature also defines a codelet as stateless, i.e., unable to inherit information from the first execution if executed twice, but this additional definition is not adopted here. Note that not all worker threads need to be codelets (atomic), and in embodiments, some or all of the workers may instead be able to communicate with each other.

[0094] Within the processing unit 10, multiple different threads from the instruction memory 12 can be interleaved through a single execution pipeline 13 (however, generally, only a subset of all threads stored in the instruction memory can be interleaved at any point in the program). The multithreaded processing unit 10 includes multiple context register files 26, each arranged to represent the state (context) of different threads running concurrently, a shared execution pipeline 13 common to the concurrently running threads, and a scheduler 24 for scheduling concurrent threads for execution through the shared pipeline in an interleaved manner, preferably a round-robin manner. The processing unit 10 is connected to a shared instruction memory 12 common to multiple threads, and also to a shared data memory 22, which is also common to multiple threads in this case.

[0095] The execution pipeline 13 includes a fetch stage 14, a decode stage 16, and an execution stage 18 which includes an execution unit capable of performing arithmetic and logical operations, address calculations, load and store operations, and other operations, as defined by the instruction set architecture. As described, in embodiments, the processing device includes multiple such execution pipelines 13 and execution units 18. However, for simplicity, Figure 1 shows only one such execution pipeline 13 and execution unit 18. Each of the context register files 26 includes a set of registers for representing the program state of each thread.

[0096] Figure 2 schematically shows an example of the registers that make up each context register file 26. Each context register file 26 includes at least one or more control registers 28, each including a program counter (PC) for each thread (to track the address of the instruction that the thread is currently executing) and, in an embodiment, a set of one or more status registers (SRs) that record the current status of each thread (such as whether it is currently executing or paused because it has encountered an error, for example). Each context register file 26 also includes each set of operand registers (OPs) 32 for temporarily holding the operands of the instructions executed by each thread, i.e., values ​​that are manipulated by the opcodes of the instructions of each thread at runtime or values ​​resulting from the operations defined by those opcodes. It will be understood that each context register file 26 may optionally include one or more other types of registers (not shown). While the term “register file” is sometimes used to refer to a group of registers in a common address space, this is not necessarily the case in this disclosure, and it should be noted that each hardware context 26 (each of the register sets 26 representing each context) may more generally contain one or more such register files.

[0097] As will be described in more detail later, the disclosed configuration has one worker context register file CX0, ..., CX(M-1) and one additional supervisor context register file CXS for each of several M threads that can run in parallel (M=3 in the illustrated example, but this is not limiting). The worker context register files are reserved to store the contexts of worker threads, and the supervisor context register file is reserved to store the context of supervisor threads. Note that in embodiments, the supervisor context is special in that it has a different number of registers than each worker. Each worker context preferably has the same number of status registers and operand registers as each other. In embodiments, the supervisor context may have fewer operand registers than each worker. Examples of operand registers that the supervisor does not have but the worker context may have: floating-point registers, accumulator registers and / or dedicated weight registers (for holding the weights of the neural network). In embodiments, the supervisor may also have a different number of status registers. Furthermore, in the embodiment, the instruction set architecture of the processing device 4 may be configured such that worker threads and one or more supervisor threads execute several different types of instructions, but also share several instruction types.

[0098] The fetch stage 14 is connected to the instruction memory 12 to fetch instructions to be executed, under the control of the scheduler 24. The scheduler 24 is configured to control the fetch stage 14 to fetch instructions sequentially from each concurrent execution thread in a set of concurrent execution threads in a repeating sequence of time slots, and as a result, divide the resources of the pipeline 13 into multiple temporally interleaved time slots, as will be described in more detail shortly. For example, the scheduling scheme may be round-robin or weighted round-robin. Another term for a processor operating in this manner is a barrel-thread processor.

[0099] In some embodiments, the scheduler 24 can access one of each thread's status registers SR, which indicates whether the thread is paused or not. Thus, the scheduler 24 controls the fetch stage 14 to fetch instructions only for the currently active thread. In embodiments, preferably, each time slot (and corresponding context register file) is always owned by one or another thread. That is, each slot is always occupied by some thread, and each slot is always included in the scheduler 24's sequence. However, there may be cases where the thread occupying any given slot is paused at that time, in which case, when the sequence reaches that slot, the instruction fetch for each thread is passed on. Alternatively, for example, in less preferred alternative implementations, it is not ruled out that some slots may be temporarily empty and excluded from the scheduled sequence. When referring to the number of time slots that an execution unit can operate on for interleaving, this refers to the maximum number of slots that executions can run on concurrently, i.e., the number supported by the unit's hardware.

[0100] The fetch stage 14 can access the program counter (PC) for each context. For each thread, the fetch stage 14 fetches the next instruction for that thread from the next address in program memory 12, as indicated by the program counter. The program counter increments each execution cycle unless branched by a branch instruction. The fetch stage 14 then passes the fetched instruction to the decode stage 16 to be decoded, and the decode stage 16 then passes the instructions for the decoded instruction, along with the decoded address of the operand register 32 specified in the instruction, to the execution unit 18 so that the instruction may be executed. The execution unit 18 can access the operand register 32 and the control register 28, which it can use when executing an instruction based on the decoded register address, such as in the case of an arithmetic instruction (for example, by adding, multiplying, subtracting, or dividing the values ​​of two operand registers and outputting the result to another operand register for each thread). Alternatively, if an instruction defines a memory access (load or store), the load / store logic of the execution unit 18 loads a value from data memory into the operand register of each thread, or stores a value from the operand register of each thread into data memory 22, in accordance with the instruction. Alternatively, if an instruction defines a branch or a status change, the execution unit changes a value in either the program counter PC or the status register SR accordingly. Note that while an instruction of a thread is being executed by the execution unit 18, instructions from the thread in the next time slot in the interleaved sequence can be decoded by the decode stage 16, and / or while an instruction is being decoded by the decode stage 16, instructions from the thread in the subsequent time slot can be fetched by the fetch stage 14 (however, in general, the scope of this disclosure is not limited to one instruction per time slot, and in alternative scenarios, for example, batches of two or more instructions may be issued from a particular thread per time slot).Therefore, interleaving favorably conceals latency in pipeline 13 according to known barrel threading techniques.

[0101] Figure 3 shows an example of an interleaving scheme implemented by scheduler 24. Here, concurrent threads are interleaved according to a round-robin scheme, so that within each round of the scheme, the round is divided into a sequence of time slots S0, S1, S2, ... for each thread to execute. Generally, each slot is the length of one processor cycle, and multiple different slots are of uniform size, but this is not necessarily the case in all possible embodiments. For example, a weighted round-robin scheme is possible in which some threads get more cycles per execution round than others. Generally, barrel threading can employ either an equal round-robin or a weighted round-robin schedule, in the latter case, the weighting can be fixed or adaptive.

[0102] Regardless of the sequence per execution round, this pattern is then repeated, with each round containing each instance of each time slot. Therefore, it should be noted that the time slots referred to herein mean the repeatedly allocated locations in the sequence, not specific instances of the time slot in a particular iteration of the sequence. In other words, the scheduler 24 allocates the execution cycles of the pipeline 13 to multiple time-interleaved (time-division multiplexed) execution channels, each execution channel containing iterations of each time slot in a repeating sequence of time slots. While four time slots exist in the illustrated embodiment, this is for illustrative purposes only, and other numbers are possible. For example, in one preferred embodiment, there are actually six time slots.

[0103] Regardless of the number of time slots into which the round-robin scheme is divided, according to this disclosure, the processing unit 10 includes one more context register file 26 than the number of time slots. That is, the processing unit 10 supports one more context than the number of interleaved time slots in which it can perform barrel threading.

[0104] For example, this is shown in Figure 2. As shown in Figure 3, when there are four time slots S0, ..., S3, there are five context register files labeled CX0, CX1, CX2, CX3, and CXS. That is, even though only four execution time slots S0, ..., S3 exist in the barrel thread scheme, and therefore only four threads can run in parallel, it is disclosed herein that a fifth context register file CXS may be added, which also includes a fifth program counter (PC), a fifth set of operand registers 32, and, in embodiments, a fifth set of one or more status registers (SR). However, as mentioned above, it should be noted that in embodiments the supervisor context may differ from the other contexts CX0, ..., CX3, and the supervisor thread may support a different set of instructions for operating the execution pipeline 13.

[0105] Each of the first four contexts, CX0, ..., CX3, is used to represent the state of each of the multiple "worker threads" currently allocated to one of the four execution time slots S0, ..., S3, in order to perform computational tasks specific to the application desired by the programmer (note that in this case, too, this may still be merely a subset of the total number of worker threads in the program, such as those stored in instruction memory 12). However, the fifth context, CXS, is reserved for a special function: to represent the state of a "supervisor thread" (SV) that has the role of coordinating the execution of worker threads, at least in the sense of assigning which worker thread W to run in which time slot S0, S1, S2, ... at any given time in the entire program. Optionally, the supervisor thread may have other "supervisors" or coordinating responsibilities. For example, the supervisor thread may be responsible for implementing barrier synchronization to ensure a specific execution order. For example, if one or more second threads depend on data output by one or more first threads running on the same processor module 4, the supervisor may perform barrier synchronization to ensure that none of the second threads are started until the first threads have finished. The supervisor may also perform barrier synchronization to ensure that one or more threads on the processor module 4 are not started until a specific external data source, such as another tile or processor chip, has completed the necessary processing to make its data available. Supervisor threads may also be used to perform other functions related to multiple worker threads. For example, a supervisor thread may be responsible for communicating data externally to the processing device 4 (to receive external data affected by one or more threads, and / or to transmit data output by one or more worker threads). In general, supervisor threads may be used to provide any kind of oversight or coordination function desired by the programmer.For example, in another scenario, the supervisor may oversee transfers between the tile local memory 12 and one or more resources in a broader system (outside the array 6), such as a storage disk or a network card.

[0106] Naturally, the four time slots are merely an example, and it should be noted that in other embodiments, if there are a maximum of M time slots 0, ..., M-1 per round, there may be other numbers, such that the processing device 4 has M+1 contexts CX, ..., CX(M-1) and CXS, i.e., one for each worker thread that can be interleaved at any given time, plus an extra context for the supervisor. For example, in one exemplary implementation, there are six time slots and seven contexts.

[0107] Referring to Figure 4, and in accordance with the teachings herein, the supervisor thread SV does not have its own time slot in the interleaved execution time slot scheme. The allocation of slots to worker threads is flexibly defined, and so do the workers. More precisely, each time slot has its own dedicated context register file (CX0, ..., CXM-1) for storing the worker context, which is used by the worker when the slot is allocated to the worker, but not when the slot is allocated to the supervisor. When a particular slot is allocated to the supervisor, that slot uses the supervisor's context register file CVS instead. Note that the supervisor can always access its own context, and workers cannot occupy the supervisor context register file CXS.

[0108] The supervisor thread SV has the ability to execute in any and all of the time slots S0, ..., S3 (or more generally, S0, ..., SM-1). The scheduler 24 is configured so that when the entire program starts, it begins by assigning the supervisor thread to all of the time slots, i.e., the supervisor SV starts execution in all of S0, ..., S3. However, the supervisor thread includes a mechanism to temporarily transfer each executing slot to the respective worker thread (for example, initially workers W0, ..., W3 in the example shown in Figure 4) at some point later (immediately or after performing one or more supervisor tasks). This is achieved by the supervisor thread executing a transfer instruction, which is referred to herein, for example, as "RUN". In embodiments, this instruction takes two operands: the address of the worker thread in instruction memory 12 and the address of data relating to that worker thread in data memory 22. RUN task_addr,data_addr

[0109] A worker thread is a portion of code that can be executed in parallel with one another, each representing one or more computational tasks to be executed. A data address can specify data that a worker thread acts upon. Alternatively, a transfer instruction may take only a single operand specifying the address of a worker thread, and the data address may be contained within the worker thread's code. Alternatively, in another example, the single operand may point to a data structure specifying the addresses of both the worker thread and the data. As previously stated, in embodiments, at least a portion of the workers may take the form of codelets, i.e., atomic units of code that can be executed in parallel. Alternatively or additionally, a portion of the workers does not have to be codelets, but may instead be able to communicate with one another.

[0110] The assign instruction ("RUN") acts on scheduler 24 to assign the current time slot in which the instruction itself is executed to the worker thread specified by the operand. Note that the assign instruction implicitly assigns the time slot in which the instruction is executed (implicit in relation to machine code instructions, this means that no operand is required to specify it, and it is implicitly understood from the opcode itself). Thus, the time slot that is assigned is the time slot in which the supervisor executes the assign instruction. In other words, the supervisor is executing in the same space it assigns. The supervisor says, "I will execute this code at this location," and from this point onward, the iterative slot is (temporarily) owned by the associated worker thread.

[0111] The supervisor thread SV performs a similar operation in each of the other time slots in the time slot to relinquish part or all of that time slot to each of the different worker threads W0, ..., W3 (selected from the larger set W0, ..., Wj in instruction memory 12). The supervisor is suspended when it has done so for the last slot (and resumes from where it left off when one of the slots is later returned by a worker W).

[0112] Therefore, the supervisor thread SV can assign multiple different worker threads, each performing one or more tasks, to different execution time slots in the interleaved execution time slots S0, ..., S3. When the supervisor thread determines that it is time to execute a worker thread, it uses a transfer instruction ("RUN") to assign this worker to the time slot in which the RUN instruction was executed.

[0113] The aforementioned "RUN" instruction is useful for starting individual worker threads when each worker thread is executing a separate codelet. In this case, each worker thread performs a different set of operations on a dataset at the memory location specified by the RUN instruction.

[0114] In this embodiment of the application example, each worker thread is configured to execute the same codelet to perform the same set of operations on different datasets placed together in an input buffer in memory. As a result of this processing by each worker, each worker produces a different set of results, each of which is stored together in an output buffer in memory. In this case, the workers can also be said to be operating in a data-parallel manner. In situations where workers are operating in a data-parallel manner, it has been found that computational efficiency can be improved by providing a corresponding execution unit of a hardware module that responds to a single instruction (referred to here as a "RUNALL" instruction) to determine one or more locations in the input buffer of data for each worker thread. In response to the execution of the RUNALL instruction, the execution unit causes the operand registers of the worker threads to be populated with one or more variables that identify the data locations for each worker, and also starts each worker in one of the slots S0, ..., S3 (or more commonly S0, ..., S(M-1)). The same one or more variables that identify the location of data for each worker are also used to allow each worker to determine one or more locations within the data output buffer for each worker thread.

[0115] In this specification, an instruction that causes one or more variables to be determined by dedicated hardware is called a RUNALL instruction, but it is sometimes also called a "first instruction".

[0116] Refer to Figure 5, which shows the elements of processing device 4 used during the execution of the RUNALL instruction. Processing device 4 contains a main pipeline 500 for the execution of specific instructions by the processing device. The main pipeline 500 is responsible for executing memory instructions, such as load and store instructions, and also for executing the RUNALL instruction. The main pipeline 500 takes the form of the execution unit 18 described above with respect to Figure 1, and includes an execution unit 510 that operates in the same manner as the execution unit 18 described above with respect to Figure 1.

[0117] The execution unit 510 includes a block of processing logic for the execution unit 510, referred to herein as a hardware module 520. The execution unit 510 also includes an instruction module 530, which includes a control circuit configured to cause the hardware module 520 to determine variables (position variables) that enable each worker thread to identify the location of the input data and the location for storing the results, in response to a RUNALL instruction. Having determined these variables, the hardware module 520 causes them to store in the operand registers 32 of the worker threads. Each worker is assigned specific data in an input buffer to operate on. Each worker performs the same set of operations in a repeating loop. The specific amount of data operated on in a single iteration of the loop is referred herein as an "input atom". An input atom may contain multiple data items that are loaded from different locations within the input atom as a result of different load instructions and are separately operated on by different arithmetic operations. Each iteration of the loop also produces a specific amount of data that is stored in a unit of memory, which is referred herein as an "output atom". The output atom may contain multiple data items generated as a result of multiple arithmetic operations and stored at different locations within the atom in response to multiple store instructions. Each iteration of the loop consumes one atom of input data and outputs one atom of output data, so the number of atoms in the input buffer and the number of atoms in the output buffer will be the same. However, the size of the input atom does not have to be the same as the size of the output atom.

[0118] The following explanation refers to determining the location in memory 22 that a worker thread accesses when executing a codelet. Accessing an input atom in memory means loading the data of that atom for use in the operation. Accessing an output atom in memory means storing the result of that atom in the output buffer. Unless otherwise specified, teachings related to "buffers" apply to both input and output buffers.

[0119] A position variable determined by the hardware module 520 for each worker thread includes an offset to the input buffer in memory 22. The offset to the input buffer in memory 22 specifies the position (from the base address of the input buffer) of the start of the first atom of data in the input buffer, which is accessed by the worker thread when executing the codelet.

[0120] Hardware module 520 directly determines the offset to the input buffer. Each worker thread, when executing its codelet, determines the offset to the output buffer in memory 22 based on the offset to the input buffer. The offset to the output buffer specifies the position (from the base address of the output buffer) of the start of the first atom of data in the output buffer, which is accessed by the worker thread when storing the result in the output buffer. The difference between the offset to the input buffer (input offset) and the offset to the output buffer (output offset) depends on the operation performed as part of the codelet. Therefore, when a worker thread executes a codelet, it determines the output offset from the input offset.

[0121] The position variable also includes the iteration count value for a particular thread, which specifies the number of atoms of data accessed by each thread in memory 22, starting from the first atom of data at the position specified by the offset value. The iteration count is the same for access to the input and output buffers.

[0122] After accessing each atom of data, the worker thread increments its address pointer, which identifies the address in the buffer being accessed. The next atom of the data being accessed is then accessed in memory 22 at the location identified by the incremented address pointer. By providing the offset and the number of atoms of the data being manipulated (in the form of an iteration count value), the variables of a particular worker thread provide the associated worker thread with enough information to identify the location where each input atom of the dataset to be processed by that worker thread and each output atom generated by that worker thread are stored.

[0123] Figure 5 shows two operand registers 32, denoted as $m0 and $m1, for a specific worker thread. These registers belong to the memory register file (MRF) 535. Although Figure 5 shows only one exemplary MRF 535 containing one $m0 register and one $m1 register, the processing device 4 contains an MRF 535 for each worker thread. The first register ($m0) stores the input offset value of the worker thread, and the second register ($m1) stores the iteration count of the worker thread. These values ​​are used by the worker thread (when the worker thread is started) to determine the location of the input atom in memory 22 and the location in memory 22 where the output atom of the result data is stored.

[0124] To derive the worker offset and iteration count values, the hardware module 520 manipulates the worker ID value, thereby enabling the determination of the buffer portion allocated to each worker. The set of operations performed by the hardware module 520 on the worker ID depends on the layout of the data in the buffer. Here are two exemplary layouts: the first is an interleaved layout in which data for multiple different workers is interleaved within the buffer, and the second is a non-interleaved layout in which the data for each worker is stored in a contiguous portion of the buffer containing data for that worker only.

[0125] In the case of an interleaved layout, the hardware module 520 determines the input offset of a particular worker as follows: Input Offset = Worker ID * Input Atom Size Equation 1

[0126] The atom size (in Equation 1 and subsequent equations) is the amount of memory space in the buffer used to store the data related to a single iteration of the loop. For the input buffer, this amount of memory (input atom size) is the amount used to store the data consumed within a single iteration of the loop when the codelet is executed.

[0127] The input offset determined by the hardware module 520 is stored in the $m0 register. Each worker thread can use the input offset value in its $m0 register to determine the output offset. Each worker thread determines its output offset from its input offset by applying a scaling factor that depends on the ratio of the input data amount to the output data amount. The output offset is given by: Output offset = Input offset * Scaling coefficient (Equation 2)

[0128] The scaling factor is determined during the compilation of the codelet and is provided as part of the codelet. The scaling factor can be used by worker threads as an intermediate operand in the RUNALL instruction to determine the output offset from the input offset.

[0129] In many cases, the scaling factor is a power of 2, such as 0.25, 0.5, 1, 2, 4, etc. Therefore, each worker thread can operate to determine the output offset by applying a left or right shift to the input offset.

[0130] Furthermore, in the case of an interleaved layer, the hardware module 520 determines the iteration count as follows:

number

[0131] Equation 3 provides the iteration counts for both the output buffer and the input buffer.

[0132] Refer to Figure 6 to illustrate how the memory space within buffer 600 (which may be an input or output buffer) can be distributed among different worker threads when data (i.e., either input data or results) is stored in buffer 600 in an interleaved layout. The memory addresses of the buffer increase vertically along the pages. Buffer 600 is shown divided into several columns, each associated with a different worker. A solid line is shown in each column, representing the offset of each worker to buffer 600, which is determined using Equation 1, and in the case of an output offset, using Equation 2. This offset for a worker is the position within buffer 600 of the first atom of data in buffer 600 that is accessed by the worker. A series of dotted lines are also shown in each column. These represent the starting position in each memory 22 of the second atom accessed by each worker thread.

[0133] The first worker thread W0 has worker ID 0, so it is assigned an offset of zero (according to Equation 1). As shown by the position of the solid line in the column for W0, W0 accesses the first atom of data in buffer 600. The second worker thread has worker ID 1, so it is assigned an offset of 1 atom. As shown by the position of the solid line in the column for W1, W1 accesses the second atom of data in buffer 600. Since there are six worker threads, each of the first six atoms of data in buffer 600 is assigned to one of the different worker threads.

[0134] If buffer 600 is the input buffer on which data items are loaded, each worker thread uses the read pointer value held in its MRF535. When loading its first data item, the read pointer value for a particular worker thread is set to be given by a combination of the base address of the input buffer and the input offset value determined for each thread according to Equation 1. After the load operation is complete, the read pointer value is incremented by the stride value to point to the next data item in the same atom. Then, before the read pointer value is incremented by the stride value, a load operation is performed to load this next data item. Each of these data items is loaded and manipulated, with the read pointer value being incremented, until the worker reaches the end of the atom. At the end of the atom, the read pointer is incremented by a further stride value to point to the next atom that the worker will access. This further stride value is greater than the stride value used to increment the read pointer between each data item in the atom. In the interleaved pattern, the additional stride value is given by multiplying the input atom size by the number of worker threads minus 1 (for example, 5 in the example shown in Figure 6) (see Equation 4 below). Loading of additional data items begins from the beginning of the next atom of data, identified by the updated read pointer value.

[0135] If buffer 600 is an output buffer where the results determined by a worker thread are stored, each worker thread uses the write pointer value held in its MRF535. The initial write pointer value for a particular thread is given by a combination of the base address of the output buffer and the output offset value determined for each thread according to equations 1 and 2. Each worker thread stores its first result data item at the position specified by its initial write pointer value. After the storage operation is complete, the write pointer value is incremented by the stride value to point to the next position (contiguous with the position of the first result data item) within the same atom. Then, before the write pointer value is incremented again by the stride value, a storage operation is performed to store the next result data item. Each result data item is stored, with the write pointer value incremented between each storage, until the worker has stored all the result data for that iteration of the loop. At the end of storing results, the write pointer is incremented by a further stride value to point to the next atom accessed by that worker. This additional stride value is greater than the stride value used to increment the write pointer between each data item within each atom. In the interleaved pattern, the additional stride value is the output atom size multiplied by the number of worker threads minus 1 (for example, 5 in the example shown in Figure 6) (see Equation 4 below). The storage of the result data for the next iteration then begins from the beginning of the next atom of data, identified by the updated write pointer value.

[0136] As shown in Figure 6, the start of the second atom accessed by thread W0 is shifted from the start of the first atom accessed by thread W0 by the atom size (either input atom size or output atom size) multiplied by the number of worker threads (i.e., 6 in this example). However, the start of the second atom accessed by thread W0 is shifted from the end of the first atom accessed by thread W0 (which is reached by applying a smaller stride value between each data item) by the atom size multiplied by the number of worker threads minus 1 (i.e., 5 in this example). Therefore, after each worker thread accesses an atom of data in buffer 600, it increments its pointer value (i.e., write pointer or read pointer) by the amount given below. Interleaved stride = Atom size * (Number of worker threads - 1) Equation 4

[0137] The interleaved stride determined according to Equation 4 is applied from the end of the first atom to determine the start of the second atom accessed by a particular worker thread. The start of the second atom is displaced from the start of the first atom by a number of atoms equal to the number of worker threads.

[0138] Access to the data atoms continues until the end of buffer 600 is reached. The iteration count for each worker thread, determined from Equation 3, allows each worker to determine when it accessed its last atom of the data. Since buffer 600 may not contain an atom number that is a multiple of the number of workers, one or more workers may have an iteration count that is one greater than the iteration count of one or more other workers. According to Equation 3, the iteration count for each worker is determined by dividing the buffer size (represented by atoms) by the number of workers. If the result of this division is not an integer (in which case each worker does not have the same iteration count), the result is rounded down to the nearest integer value. If the worker ID of each worker is smaller than the remainder when the buffer size is divided by the number of workers, a value of 1 is added to the rounded result.

[0139] Refer to Figure 7, which illustrates an example of how atoms for different workers (i.e., input atoms or output atoms) may be arranged within buffer 600, and how workers adjust their pointers (i.e., read pointers or write pointers) according to the stride value between accessing each atom of data. In Figure 7, each numbered element represents an atom. The atoms are shown with addresses increasing horizontally across the page. In practice, buffer 600 is likely to contain more atoms than the number shown in Figure 7. The numbers shown in each element represent the ID of the worker accessing the atom of data.

[0140] The arrows illustrate the stride behavior of the first worker (W0) and the second worker (W1), but other workers exhibit similar stride behavior. The stride behavior between positions within each atom is not shown. However, accessing each atom almost always involves loading (in the case of input atoms) or storing (in the case of output atoms) multiple data items. After accessing a data atom in memory, each worker increments its pointer by the stride value determined from Equation 4 (i.e., 5 atoms in this example), and then accesses the data atom pointed to by the updated pointer. In the example in Figure 7, the number of atoms in buffer 600 is such that the number of atoms in buffer 600 is not a multiple of the number of workers, but one greater than a multiple of the number of workers (because worker W0 has one more atom associated with it). As a result, the iteration count of worker W0 is one greater than the iteration count of the remaining workers, and therefore worker W0 iterates the codelet loop one more time.

[0141] In the case of a separate layout, the hardware module 520 determines the input offset of a particular worker as follows:

number

[0142] In the case of the interleaved operation described above, the output offset is determined by the worker thread applying a scaling factor to the input offset. Thus, the output offset is determined by applying Equation 2 (wherein the input offset is given by Equation 5).

[0143] As in the case of interleaving, the iteration count for accessing the input buffer is the same as the iteration count for accessing the output buffer. To determine the iteration count per worker, the hardware module 520 determines a value (referred to herein as "max_count") that represents the maximum iteration count for any one worker. max_count is given by:

number

[0144] Here, the buffer size is given in terms of the number of atoms in the buffer. If the number of atoms in the buffer is less than or equal to the number of workers, the value of max_count is 1. Otherwise, the value of max_count is 2 or greater, because at least one worker accesses two or more atoms in the buffer.

[0145] Furthermore, in order to determine the iteration count for each worker, the hardware module 520 determines a worker ID value such that workers with an ID value smaller than this determined value have an iteration count given by the max_count value, while workers with an ID value greater than or equal to the determined value have a worker ID value with a lower iteration count. This determined worker ID value is referred to herein as the "cutoff" and is given by the following:

number

[0146] Here, the buffer size is given in terms of the number of atoms in the buffer. If the worker ID is less than this cutoff value, the worker is assigned an iteration count equal to the max_count value. If the worker ID is equal to this cutoff value, the iteration count is set to a different value (referred to herein as the “remainder”), which is given by the remainder of the division performed in Equation 7, namely as follows: Remainder = Buffer size % max_count Expression 8

[0147] If the worker ID is greater than the cutoff value, the iteration count is set to equal to zero.

[0148] Therefore, to determine the iteration count of a worker thread, the hardware module 520 determines a cutoff value according to Equation 7. Next, for each worker thread, the hardware module 520 compares the cutoff value with the worker ID value for that worker thread. If the worker ID value is less than the cutoff value, the hardware module 520 sets the iteration count to be equal to max_count. If the worker ID is equal to the cutoff value, the hardware module 520 sets the iteration count of that worker to be equal to the remainder determined using Equation 8. If the worker ID value is greater than the cutoff value, the hardware module 520 sets the iteration count of that worker to be equal to zero.

[0149] Refer to Figure 8A, which shows how the memory space of buffer 800 (which may be an input buffer or an output buffer) can be shared among different worker threads, assuming each worker is allocated a contiguous region of memory. The buffer 800 shown in Figure 8A represents an example where buffer 800 is equally divided among worker threads, so that each worker thread is allocated a memory region of equal size to store its associated data (i.e., input data or result data).

[0150] The input offset for each worker thread is determined by the hardware module 520 applying Equation 5. The output offset is further determined by each worker thread applying Equation 2 to the input offset determined using Equation 5. As a result, the offset value specifying the position of the worker thread's first atom will be located either at the beginning of the buffer (for worker W0) or immediately after the memory region allocated to another worker. As an example, Figure 8A shows the offset value for worker W2. This offset value indicates the position of the first atom 810 in buffer 800, which is accessed by worker thread W2 when executing a codelet. This atom is located adjacent to the last atom 820 in buffer 800, which is accessed by worker thread W1 when executing a codelet.

[0151] Throughout the execution of the codelet, each worker accesses each atom in the allocated memory space within buffer 800. For input buffers, when accessing each input atom, the worker loads each of the data items belonging to one of those atoms. During each load, a stride is applied to its read pointer. When the worker loads the last data item belonging to one of its input atoms, the same stride may be applied to the read pointer to point to the data item at the beginning of the next atom, since those atoms are contiguous. Each worker continues to apply a stride to its read pointer until it loads the last item of the last atom in its allocated memory region.

[0152] In the case of an output buffer, when a worker accesses each output atom, it stores multiple data items in that output atom. A stride is applied to its write pointer during each storage. When the worker stores the last data item belonging to one of its output atoms, the same stride may be applied to the write pointer to point to the data item at the beginning of the next atom, since those atoms are contiguous. Each worker continues to apply a stride to its write pointer until it stores the last data item in the last atom in its allocated memory region.

[0153] Each worker continues to access atoms in buffer 800 until its counter reaches its iteration count. The iteration count for each worker is determined by the hardware module 520 applying equations 6-8. In the example in Figure 8A, the total number of atoms in buffer 800 is a multiple of the number of workers. In this case, the hardware module 520 determines a cutoff value that is equal to the number of workers. Since worker IDs are assigned in the range of 0 to n-1 (where n is the number of workers), each worker has a worker ID value below the cutoff value. Therefore, the hardware module 520 assigns each worker thread the same iteration count given by the max_count value. As a result, the worker memory areas in Figure 8A are of equal size.

[0154] Refer to Figure 8B, which shows an alternative example of how buffer 850 can be divided among worker threads. In this example, the number of atoms in buffer 850 is not a multiple of the number of worker threads, and as a result, the iteration count of the last worker thread (i.e., worker W5 in this example, the worker thread with the largest offset value) is smaller than the iteration counts of the other worker threads. When determining the iteration count, hardware module 520 determines that the cutoff value is equal to n-1 (where n is equal to the number of workers), and therefore equal to the worker ID of the last worker. As a result, hardware module 520 assigns each worker (except the last worker) an iteration count value equal to the max_count value determined from equation 6. Furthermore, hardware module 520 assigns the last worker an iteration count value equal to the remainder value determined from equation 8.

[0155] Refer to Figure 9, which illustrates an example of how data atoms may be accessed by two different worker threads in either a portion 900 of buffer 800 or 850. While this figure shows data atoms being accessed by a first worker thread (W1) and a second worker thread (W2), the same applies to any worker thread allocated adjacent memory regions within buffers 800 and 850. Worker W1's data items are labeled with the letters A, B, or C, and worker W2's data items are labeled with the letters D, E, or F. The data items may constitute input data loaded from buffer 900 (if buffer 900 is an input buffer) for processing by the worker threads associated with them. The data items may also constitute result data stored in buffer 900 (if buffer 900 is an output buffer) for processing by the worker threads associated with them. The data items are shown grouped by atoms, with data items associated with a particular atom being labeled with the same letter A through F, respectively.

[0156] Each worker loads or stores one or more data items within a specific time slot of the barrel thread processor before control is passed to the next worker. Here, we describe an example where each worker accesses only a single data item within each time slot. However, it will be understood that the number of items accessed within a single time slot can vary.

[0157] First, worker W1 executes an instruction to load or store data item A1 at a memory location identified by its offset value, provided by the hardware module 520. Next, worker W1 updates its pointer value to a new value by applying a stride to the pointer. The stride value is given by the size of the data item. Before worker W1 loads or stores the next data item A2, worker W2 executes an instruction to load or store its first data item B1 from a location in memory 22 identified by its offset value, provided by the hardware module 520. Next, worker W1 updates its pointer value to a new value by applying a stride to the pointer, the stride value being given by the size of the data item. When its allocated time slot is selected in the barrel thread processor, worker W1 loads or stores the next data item A2. Similarly, when its allocated time slot is selected in the barrel thread processor, worker W2 loads or stores the next data item B2. In this way, the loading or storage of data items by different workers is interleaved in time. Workers W1 and W2 each continue loading or storing data items until the number of completed iterations of the worker's operation loop equals the worker's iteration count, taking a stride between each load / storage.

[0158] While loading different data items, each worker thread may perform one or more arithmetic operations on the loaded data items and one or more storage operations to store the results of those arithmetic operations in memory 22.

[0159] Returning to Figure 5, it shows how the hardware module 520 receives different variables from the registers of the processing device 4, and how the hardware module 520 outputs variables (i.e., input offset and iteration count) that are accessible to worker threads and used by those worker threads to perform load operations from input buffers 700, 800, and 850 and storage operations to output buffers 700, 800, and 850 in memory 22.

[0160] As shown in the figure, one of the supervisor control and status registers (CSR) is the RUNALL control register 545. The RUNALL control register 545 stores specific values ​​used by the hardware module 520 in response to the execution of the RUNALL instruction. See Figure 10 for the contents of the RUNALL control register 545. This register 545 stores a value (indicated as TSIZE) that represents the total number of atoms in the input buffer containing the worker data. The register 545 also stores a value (indicated as ATOM) that represents the size of each atom in the data in the input buffer. The size of each atom is given by the square of the value of ATOM. The register 545 also stores a value (indicated as PRECALC) that selects which calculation is performed by the hardware module 520 to determine the worker position variables. PRECALC can be used to specify at least whether interleaved mode is applied (in which case the hardware module 520 can determine the position variables according to equations 1 and 3) or whether continuous mode is applied (in which case the hardware module 520 can determine the position variables according to equations 5-8). As shown in the diagram, some bits of the RUNALL control register 545 (bits 23-29 in the illustrated example) may be reserved to add additional functionality.

[0161] The hardware module 520 can access values ​​from the RUNALL control register and, in response to the execution of the RUNALL instruction by the execution unit 510, can use these values ​​to determine the position variables. The hardware module 520 also accesses the worker ID value of each worker thread located in the register (shown as Worker.$wsr.CTXTID_M1 in Figure 5) and uses this to determine the position variables as well. As can be understood from equations 1-8, in order to determine the position variables, the hardware module 520 also uses a value representing the number of worker threads. This number (6 in the example described, but could be another number) is implicit in the RUNALL instruction itself because it is specific to the architecture of the processing device 4.

[0162] In response to the execution of the RUNALL instruction, the hardware module 520 examines the mode operation (i.e., PRECALC) to determine whether interleaved mode or continuous mode is applied. If interleaved mode is applied, the hardware module 520 uses the worker ID value, atom size, and input buffer size to determine the input offset and iteration count for each worker according to equations 1 and 3. If interleaved mode is applied, the hardware module 520 uses the worker ID value, atom size, and input buffer size to determine the offset and iteration count for each worker according to equations 5-8. In either case, the circuitry of the processing device 4 causes each input offset value to be written to the $m0 register of the associated worker thread, and each iteration count value to be written to the $m1 register of the associated worker thread.

[0163] Hardware module 520 also receives an address value from the register of the MRF540 associated with the supervisor thread (indicated as $mVERTEX_BASE). This address value is a pointer to a structure in memory 22 that contains the base addresses of the input buffer and the output buffer. The input buffer base address specifies the starting address value of the input buffer. Similarly, the output buffer base address specifies the starting address value of the output buffer. These base address values ​​are the same for all worker threads. Each individual input offset value determined by hardware module 520 for each worker thread is added to the base address value by each worker thread to provide the address of the first atom of the data in the input buffer to be accessed. Similarly, each individual output offset value determined by hardware module 520 for each worker thread is added to the base address value by each worker thread to provide the address of the first atom of the data in the output buffer to be accessed. In response to the execution of the RUNALL instruction, hardware module 520 causes each worker thread's CSR (indicated as Worker.$VERTEX_BASE) to write a pointer to the structure containing these base addresses. During execution, each worker thread then loads the base addresses of the input and output buffers from the location identified by the pointer in its CSR.

[0164] As described, the main pipeline includes execution units for performing certain types of operations, including load and store operations, adding data to the MRF register 535 in response to a RUNALL instruction, and performing integer arithmetic operations. Processing device 4 also includes an auxiliary pipeline 550, which includes execution units for performing floating-point arithmetic operations on the values ​​of data loaded into the arithmetic register file (ARF) 555. The load operation described above may be performed to load values ​​from the input buffer into the registers of the ARF 555. These values ​​may then be used in the arithmetic operations performed by the execution units of device 4 to generate result values. The result values ​​are then returned to the output buffer and stored in the manner described.

[0165] The above description assumes that many operations are performed by worker threads. Unless otherwise specified, all operations described as being performed by worker threads are performed by each worker thread, and each worker thread performs the same operation but manipulates the data associated with itself.

[0166] The embodiments described above are merely examples.

[0167] Refer to Figure 11, which shows an example of a hardware module 520 according to an embodiment. The processing circuit 1220 performs the operations described above, as performed by the hardware module 520. The hardware module 520 includes one or more input buffers 1210 for receiving variables (e.g., PRECALC, TSIZE, ATOMSIZE, worker ID) used to calculate position variables (e.g., offset and iteration count). These variables are supplied from the input buffers 1210 to the processing circuit 1220, which uses them to derive the position variables in the manner described above. The processing circuit 1220 includes any suitable logic circuitry for this purpose. The hardware module 520 includes one or more output buffers 1230 for storing the results of the processing performed by the processing circuit 1220. These results include position variables provided by the processing device 4's circuitry to the relevant context registers of the processing device 4 (e.g., $m0 and $m1).

[0168] Refer to Figure 12, which shows Method 1200 according to an exemplary embodiment. Method 1200 allows each worker thread to locate its input data in memory.

[0169] In S1210, in response to the execution of the first instruction, for each worker thread, based on the identifier of each worker thread, the hardware module 520 determines one or more variables that enable each worker thread to find its dataset in the input buffer which should be manipulated by each worker thread.

[0170] In S1220, in response to the execution of the first instruction, the processing device circuitry causes at least one of the operand registers associated with each worker thread to input one or more variables determined for each worker thread. Each operand register is associated with one of the worker threads and is configured to store at least one operand for at least one instruction executed by that associated worker thread.

[0171] In S1230, at least one execution unit interleaves the execution of multiple worker threads.

[0172] Refer to Figure 13, which shows Method 1300 according to an exemplary embodiment. Method 1300 allows each worker thread to identify the location in memory where the resulting data will be stored.

[0173] In S1310, in response to the execution of the first instruction, for each worker thread, based on the identifier of each worker thread, the hardware module 520 determines one or more variables that enable each worker thread to determine the location in the output buffer where its respective result set is written.

[0174] In S1320, in response to the execution of the first instruction, the hardware module 520 causes at least one operand register associated with each worker thread to be populated with one or more variables determined for each worker thread. Each operand register is associated with one of the worker threads and is configured to store at least one operand for at least one instruction executed by that associated worker thread.

[0175] In S1330, at least one execution unit interleaves the execution of multiple worker threads.

[0176] It will be understood that the embodiments are described merely as examples.

Claims

1. At least one execution unit configured to interleave the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different datasets held in the input buffer of the processing device's memory, A plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by one of the associated worker threads, Includes, The at least one execution unit, in response to a first instruction executed by the at least one execution unit before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, one or more variables are determined that enable the worker thread to find the dataset in the input buffer that should be manipulated by the worker thread. To cause at least one of the operand registers associated with the worker thread to be input with the one or more variables determined for the worker thread, A processing device including a hardware module containing a processing circuit configured to perform a certain task.

2. The processing device according to claim 1, wherein for each of the worker threads, the one or more variables include an offset to the input buffer in which at least one item of the worker thread's dataset is stored.

3. The processing device according to claim 2, wherein each of the worker threads is configured to execute a load instruction to load an item of the dataset from a memory location given by the offset with respect to the worker thread.

4. The processing device according to claim 2, wherein for each of the worker threads, at least one item of the worker thread's dataset includes a first item which should be accessed first in the sequence by the worker thread.

5. The processing device according to claim 1, wherein for each of the worker threads, the operation includes the same set of operations that are repeatedly performed in a loop, and each of the worker threads is configured to perform each iteration of the loop on one of a plurality of subsets of the dataset.

6. The processing device according to claim 5, wherein for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be executed by the worker thread, and each of the worker threads is configured to continue loading data from the input buffer until the number of completed iterations of the loop by the worker thread matches the iteration count of the worker thread.

7. The processing device according to claim 5, wherein each of the worker threads is configured to increment a pointer to the next starting address of the subset of the dataset between each iteration of the loop when executing the same set of code.

8. The processing device according to claim 1, further comprising, for each of the worker threads, a further register for storing control information that enables the processing circuit to determine one or more variables of the worker thread.

9. The processing device according to claim 8, wherein the execution unit is configured to execute a separate instruction for loading the control information into a further register before executing the first instruction.

10. The processing circuit is configured to receive a set of input information for each of the worker threads, which enables the processing circuit to determine one or more variables of the worker thread, and the input information is The size of the input buffer in which each of the worker threads' datasets is stored, and The worker thread is configured to perform a single iteration of the same subset of the operations that are repeated in the loop, with a size for each subset of the dataset. The processing device according to claim 1, comprising one or more of the above.

11. The processing device according to claim 10, wherein the processing circuit is configured to determine, for each of the worker threads, an iteration count indicating the number of iterations of the loop to be executed by the worker thread, based on the size of the input buffer and the number of worker threads.

12. The processing circuit, for each of the worker threads, The size of each subset of the dataset, The number of worker threads and, The processing device according to claim 10, configured to determine an offset to the input buffer in which at least one item of the worker thread's dataset is stored, based at least on the above.

13. The processing device according to claim 10, wherein the input information includes control information that enables the processing circuit to determine one or more variables of the worker thread.

14. The processing device according to claim 1, wherein the processing circuit is configured to receive instructions relating to one of a set of data layout modes for the input buffer, and to select a set of calculations to be performed to determine one or more variables based on the instructions for one of the set of data layout modes.

15. The aforementioned data layout mode is Interleaved mode in which data belonging to different worker threads is interleaved in the input buffer, and For each worker thread, the worker thread's dataset is in a continuous mode where it is located in a single contiguous region of the input buffer. The processing device according to claim 14, comprising at least one of the following.

16. An execution unit configured to interleave the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different input datasets, generating different result sets, and storing the result sets of said operations in an output buffer in the memory of the processing device, A plurality of operand registers, each associated with one of the worker threads and configured to store at least one operand for at least one instruction executed by that associated worker thread, Includes, The at least one execution unit, in response to a first instruction executed by the at least one execution unit before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, the worker thread determines one or more variables that enable it to determine the location in the output buffer where the result set is written, To cause at least one of the operand registers associated with the worker thread to be input with the one or more variables determined for the worker thread, A processing device including a hardware module containing a processing circuit configured to perform a certain task.

17. The processing device according to claim 16, wherein for each of the worker threads, the one or more variables include a memory address value that enables the worker thread to identify one of the locations in the output buffer where a portion of the result set is written.

18. The processing device according to claim 17, wherein for each of the worker threads, the memory address value is an offset value to an item of input data for the worker thread, and each of the worker threads is configured to identify one of the positions in the output buffer by scaling the memory address in order to obtain the offset value to the output buffer.

19. The processing device according to claim 18, wherein each of the worker threads is configured to execute a storage instruction to store an item of the result set in a memory location identified by the offset value of the worker thread.

20. The processing device according to claim 16, wherein for each of the worker threads, the operation includes the same set of operations that are repeatedly performed in a loop, and each of the worker threads is configured to perform the iterations of the loop to produce different subsets of the result set.

21. The processing device according to claim 20, wherein for each of the worker threads, the one or more variables include an iteration count indicating the number of iterations of the loop to be performed by the worker thread, and each of the worker threads is configured to continue storing the result set until the number of completed iterations of the loop by the worker thread matches the iteration count of the worker thread.

22. The processing device according to claim 20, wherein each of the worker threads is configured to increment a pointer between each iteration of the loop to point to the starting address of the next portion of the memory in which the next subset of the result set is to be stored, when executing the same set of code.

23. The processing device according to claim 16, further comprising, for each of the worker threads, a further register for storing control information that enables the processing circuit to determine one or more variables of the worker thread.

24. The processing device according to claim 23, wherein the at least one execution unit is configured to execute a separate instruction for loading the control information into a further register before executing the first instruction.

25. The processing circuit is configured to receive a set of input information for each of the worker threads, which enables the processing circuit to determine one or more variables of the worker thread. The input information includes one or more of the sizes of the input buffers containing each of the different input datasets, and the sizes of each of the multiple subsets of the result set. The processing device according to claim 16, wherein one of the worker threads is configured to generate one of the subsets in response to repeatedly performing a single iteration of the same set of operations in a loop.

26. The processing device according to claim 25, wherein the processing circuit is configured to determine, for each of the worker threads, an iteration count indicating the number of iterations of the loop to be executed by the worker thread, based on the size of the input buffer and the number of worker threads.

27. The processing circuit, for each of the worker threads, The processing device according to claim 25, configured to determine a memory address value that enables the worker threads to identify one of the locations in the output buffer where a portion of the result set is written, based at least on the size of the subset and the number of worker threads.

28. The processing device according to claim 25, wherein the input information includes control information that enables the processing circuit to determine one or more variables of the worker thread.

29. The processing device according to claim 16, wherein the processing circuit is configured to receive an instruction relating to one of a set of data layout modes for the output buffer, and to select a set of calculations to be performed to determine one or more variables based on the instruction for one of the set of data layout modes.

30. The aforementioned data layout mode is The processing device according to claim 29, comprising at least one of an interleaved mode in which result sets belonging to different worker threads are interleaved in the output buffer, and a continuous mode in which, for each worker thread, the result set of the worker thread is stored in a single contiguous area of ​​the output buffer.

31. The processing device according to claim 16, wherein at least one execution unit is arranged to execute a supervisor thread in addition to the plurality of worker threads, and the supervisor thread is configured to execute the first instruction before the worker threads are started.

32. The processing device according to claim 31, wherein the first instruction starts the worker thread.

33. The processing device according to claim 31, wherein a further register that stores control information enabling the processing circuit to determine the one or more variables of the worker thread is a register of the supervisor thread.

34. The processing device according to claim 16, wherein the execution unit is configured to execute each of the worker threads in a repeating sequence, the sequence comprising a plurality of time slots on which at least one execution unit can operate to interleave the execution of the worker threads.

35. The execution unit is Before the worker thread is started, a supervisor thread is executed in each of the different time slots, configured to execute the first instruction before the worker thread is started. In response to the execution of the first instruction, the supervisor thread is enabled to transfer each of the time slots in which it is running to one of the worker threads, The processing device according to claim 34, configured to perform the following:

36. The at least one execution unit includes a plurality of execution units, the first execution unit of which includes the hardware module and is configured to execute the first instruction. The processing device according to claim 16, wherein the second execution unit among the plurality of execution units is configured to perform arithmetic operations on the dataset loaded from the memory.

37. Interleaving the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different datasets held in the input buffer of the processing device's memory, In response to a first instruction executed before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, one or more variables are determined that enable the worker thread to find the dataset in the input buffer that should be manipulated by the worker thread. To cause at least one operand register associated with the worker thread to be input with the one or more variables determined for the worker thread, A method that includes this.

38. A non-temporary computer-readable medium for storing a computer program including an executable instruction set that causes at least one processor to execute a method, wherein the method is Interleaving the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different datasets held in the input buffer of the processing device's memory, In response to a first instruction executed before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, one or more variables are determined that enable the worker thread to find the dataset in the input buffer that should be manipulated by the worker thread. The worker thread is made to input one or more variables determined for the worker thread into at least one operand register of the worker thread, Non-temporary computer-readable media, including [specific examples of such media].

39. Interleaving the execution of multiple worker threads, each configured to execute the same set of code to perform operations on different input datasets, generate different result sets, and store those result sets in an output buffer in the processing device's memory, In response to a first instruction executed before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, the worker thread determines one or more variables that enable it to determine the location in the output buffer where the result set is written, To cause at least one operand register associated with the worker thread to be input with one or more variables determined for the worker thread, A method that includes this.

40. A non-temporary computer-readable medium for storing a computer program including an executable instruction set that causes at least one processor to execute a method, wherein the method is Interleaving the execution of multiple worker threads configured to run the same set of code to perform operations on different input datasets, generate different result sets, and store the result sets in an output buffer in the processing device's memory, In response to a first instruction executed before the execution of the same code set, for each of the worker threads, Based on the identifier of the worker thread, the worker thread determines one or more variables that enable it to determine the location in the output buffer where the result set is written, To cause at least one operand register associated with the worker thread to be input with one or more variables determined for the worker thread, Non-temporary computer-readable media, including [specific examples of such media].

Images