Processing device using variable stride patterns

Abstract

In certain applications, some of the application data held in the memory of the processing device (e.g., generated as a result of operations performed by the execution unit) is arranged in a regular repeating pattern in the memory, and therefore the execution unit may set a stride pattern suitable for use by the transmit engine. To access multiple data items arranged together in a regular pattern, the transmit engine accesses the memory at locations according to the configured stride pattern. As is done for transmission, the execution unit may set a stride pattern for use by the receive engine. Upon receiving multiple data items, the receive engine stores the data items in locations in memory determined according to the configured stride pattern.

Description

[Technical Field]

[0001] This application relates to processing devices that include a memory, and in particular to processing devices that are configured to send and receive data to and from identified locations within that memory. [Background technology]

[0002] A processing device may include an execution unit and a memory. The execution unit may execute one or more program threads to perform operations on data loaded from the memory and produce results, which are stored in the memory. The results may be subject to subsequent processing by the execution unit or may be dispatched from the processing device.

[0003] To increase the processing power available to a running application, multiple processing units can be connected and configured to perform multiple different types of operations in parallel and then exchange their results. One relevant example of the usefulness of such parallel processing architectures can be found in machine intelligence. As is well known to those skilled in the art of machine intelligence, machine intelligence algorithms are based on making iterative updates to a "knowledge model," which can be represented by a graph of multiple interconnected nodes. Each node represents a function of its inputs. Some nodes receive inputs to the graph, and some nodes receive inputs from one or more other nodes, while the outputs of some nodes form the inputs of other nodes and the outputs of some nodes provide the output of the graph (in some cases, a given node may even have all of the inputs to the graph, outputs from the graph, and connections to other nodes). Furthermore, the function at each node is parameterized by one or more individual parameters, e.g., weights. During the learning phase, the goal is to find values ​​for various parameters based on a set of empirical input data such that the graph as a whole produces a desired output for a set of possible inputs. Various algorithms for doing this are known in the art, such as backpropagation algorithms based on stochastic gradient descent. Over multiple iterations based on the input data, the parameters are gradually adjusted to reduce the error, and the graph converges toward a solution. In subsequent stages, the trained model can be used to make predictions of outputs given a specified set of inputs, or to make inferences about inputs (causes) given a specified set of outputs.

[0004] In such machine learning applications, the implementation of each node involves processing data, and the interconnections of the graph correspond to the data exchanged between the nodes. Typically, at least some of the processing of each node can be performed independently of some or all of the other nodes in the graph; therefore, large graphs offer great opportunities for parallelization.

[0005] When running an application that involves multiple processing devices, it is typically necessary to exchange data between those devices. When sending data, a processing device must include the functionality to identify where in its memory each data item that is to be sent is stored. Additionally, when receiving data, a processing device must include the functionality to identify where in its memory each received data item is to be stored. Summary of the Invention [Problem to be solved by the invention]

[0006] To allow a processing device to identify locations in memory for transmitting or storing data, memory addresses can be explicitly identified within a compiled code set stored within the processing device. However, explicitly storing all memory addresses within the processing device's instruction memory requires a significant amount of storage space within the instruction memory. It would be desirable to find a way to reduce the amount of storage space required while still being able to identify memory addresses for transmitting and storing data. [Means for solving the problem]

[0007] According to a first aspect, a processing device includes a memory, an execution unit configured to execute transmit instructions for transmitting data external to the processing device, and a transmit engine. The transmit engine includes one or more registers for storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which the memory is accessed to transmit data external to the processing device, and a processing circuit. The processing circuit is configured to, in response to execution of one or more instances of the transmit instructions by the execution unit, cause multiple data items at locations in the memory identified by the pointer to be transmitted to at least one destination external to the processing device, and update the pointer according to the variable stride pattern between each transmission of one of the multiple data items. The execution unit is configured to execute at least one instruction for updating the variable stride pattern used by the transmit engine, wherein updating the variable stride pattern includes, in response to further instances of the transmit instructions executed by the execution unit, updating at least one of the one or more configuration values ​​for use by the transmit engine in making one or more subsequent transmissions.

[0008] The inventors have recognized that in certain applications, portions of application data held in the memory of a processing unit (e.g., generated as a result of operations performed by an execution unit) may be arranged in a regular, repeating pattern in the memory, thereby allowing the execution unit to set an appropriate stride pattern for use by the transmit engine. To access multiple data items arranged together in a regular pattern, the transmit engine accesses the memory at locations according to the configured stride pattern. As a result, the transmit engine can transmit data from the appropriate location in memory without requiring the execution unit to provide explicit address values ​​for every data item being transmitted from the processing unit. In this manner, embodiments provide a method that can reduce the size of code for transmitting data.

[0009] In some embodiments, the at least one instruction includes an instruction having a combined send and update function that, when executed by the execution unit, both provides at least one new value of one or more configuration values ​​to the transmit engine for use by the transmit engine in making one or more subsequent transmits, and transmits a further data item from memory.

[0010] In some embodiments, the at least one instruction, when executed by the execution unit, causes the transmit engine to copy at least one new value of one or more of the configuration values ​​into a portion of one or more registers while continuing to update a pointer using at least one previous value of the one or more configuration values.

[0011] In some embodiments, the execution unit is configured to cause the new values ​​to replace previous values ​​for use by the transmit engine following copying the new values ​​into a portion of one or more registers.

[0012] In some embodiments, at least one of the one or more configuration values ​​includes at least one of a stride size value that defines the separation in memory between two data items that are successively transmitted by the transmit engine and a repetition value that defines the number of times a stride of a particular size is applied by the transmit engine before applying a stride of a different size.

[0013] In some embodiments, the memory is configured to store a compiled set of code including one or more instances of transmit instructions and one or more instances of at least one instruction to update a variable stride pattern, the compiled set of code configured to execute each of the one or more instances of the transmit instructions during an allocated clock cycle that is a predetermined number of processor clock cycles after receipt of a synchronization signal, and the compiled set of code configured to execute each of the one or more instances of the at least one instruction to update the variable stride pattern during an allocated clock cycle that is a predetermined number of processor clock cycles after receipt of the synchronization signal.

[0014] In some embodiments, the processing unit includes wiring configured to carry further data along a loopback path from the memory to be stored back in the memory, and the processing circuitry of the transmit engine is configured to, in response to execution of one or more additional instances of the transmit instruction by the execution unit, cause the plurality of items of further data to be rearranged in the memory by transmitting the plurality of items of further data via the loopback path in an order determined by a current variable stride pattern active for the transmit engine.

[0015] According to a second aspect, a processing device includes a memory, an execution unit, and a receive engine. The receive engine includes one or more registers for storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which the memory is accessed to store data received by the processing device, and a processing circuit. The processing circuit is configured to, in response to reception of a plurality of data items at the processing device, store the plurality of data items at locations in the memory identified by the pointer, and update the pointer according to the variable stride pattern for each reception and storage of one of the plurality of data items. The execution unit is configured to execute at least one instruction for updating the variable stride pattern used by the receive engine, wherein updating the variable stride pattern includes updating at least one of the one or more configuration values ​​such that the new value of the at least one of the one or more configurations is used by the receive engine to store one or more further data items received by the processing device.

[0016] Similar to what is done for transmission, the execution unit may configure a stride pattern for use by the receive engine. Upon receiving multiple data items, the receive engine stores the data items in locations in memory determined according to the configured stride pattern. The data items may be stored in a regular, repeating pattern suitable for use by the execution unit to perform operations on the received data. The receive engine can store received data in appropriate locations in memory using the configured stride pattern without requiring the execution unit to provide explicit address values ​​for every data item received at the processing unit. In this manner, embodiments provide a method that can reduce the size of code for storing received data.

[0017] The first and second aspects represent closely related alternative solutions to the same technical problem.

[0018] In some embodiments, the at least one instruction includes an instruction with a combined send and update function that, when executed by the execution unit, causes both: providing at least one new value of one or more of the configuration values ​​to the receiving engine for use by the receiving engine in storing one or more further data items; and transmitting the output data item from memory.

[0019] In some embodiments, the at least one instruction, when executed by the execution unit, causes the receiving engine to copy at least one new value of the one or more configuration values ​​into a portion of one or more registers while continuing to update the pointer with the at least one previous value of the one or more configuration values.

[0020] In some embodiments, the execution unit is configured to cause the new values ​​to replace previous values ​​for use by the receive engine following copying the new values ​​into a portion of one or more registers.

[0021] In some embodiments, at least one of the one or more configuration values ​​includes at least one of a stride size value that defines a separation in memory between locations in memory where data items are stored consecutively by the receiving engine and a repetition value that defines the number of times a stride of a particular size is applied by the receiving engine before applying a stride of a different size.

[0022] In some embodiments, the processing unit includes wiring configured to transmit the further data from the memory along a loopback path to be stored back in the memory, and the processing circuitry of the receive engine is configured to, in response to receiving the plurality of items of further data via the loopback path, cause the plurality of items of further data to be rearranged in the memory by storing the plurality of items of further data in an arrangement that is dependent on the current variable stride pattern active for the receive engine.

[0023] In some embodiments, the memory is configured to store a compiled code set including one or more instances of at least one instruction for updating a variable stride pattern, the compiled code set being configured such that each of the one or more instances of the at least one instruction is executed by the execution unit during an allotted clock cycle that is a predetermined number of processor clock cycles after receipt of a synchronization signal at the processing unit.

[0024] In some embodiments, at least one of the one or more configuration values ​​includes multiple configuration values.

[0025] In some embodiments, the execution unit is configured, upon execution of the at least one instruction, to update a string of bits held in a configuration register of the processing unit, the string of bits held in the configuration register being divided into a plurality of subsets of bits, each subset of bits mapping to one of a plurality of configuration values, such that, in response to writing the string of bits to the configuration register, the processing circuitry is configured to use the updated value for the one of the plurality of configuration values ​​to which the subset is mapped.

[0026] In some embodiments, each of the multiple subsets of bits is shorter in length than its associated one of the multiple configuration values ​​to which it is mapped.

[0027] In some embodiments, the plurality of configuration values ​​includes a plurality of stride size values.

[0028] In some embodiments, the plurality of configuration values ​​includes a plurality of repetition values, each of which is associated with one of the stride size values ​​and specifies the number of times that the associated one of the stride size values ​​is applied before the processing circuitry applies a different one of the stride size values.

[0029] In some embodiments, the one or more registers are control and status registers that belong to a thread executing on an execution unit.

[0030] According to a third aspect, there is provided a multi-tile processing unit comprising a plurality of instances of a processing device according to the first aspect or the second aspect, wherein each of the plurality of instances of the processing device is a tile.

[0031] According to a fourth aspect, there is provided a method of transmitting data from a processing device, the method comprising: storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which memory is accessed to transmit data external to the processing device; and, in response to execution of one or more instances of a send instruction, causing transmission of a plurality of data items at locations in the memory identified by the pointer to at least one destination external to the processing device, updating the pointer in accordance with the variable stride pattern between each transmission of one of the plurality of data items. The method further comprises executing at least one instruction for updating the variable stride pattern, wherein updating the variable stride pattern comprises, in response to execution of further instances of the send instruction, updating at least one of the one or more configuration values ​​for use in making one or more subsequent transmissions.

[0032] In some embodiments, the at least one instruction includes instructions having a combined transmit and update function, and the method includes executing the instructions having the combined transmit and update function to both provide at least one new value of the one or more configuration values ​​to the transmit engine for use by the transmit engine in making one or more subsequent transmissions, and to transmit a further data item from the memory.

[0033] In some embodiments, the at least one instruction, when executed by the execution unit, causes the transmit engine to copy at least one new value of one or more of the configuration values ​​into a portion of one or more registers while continuing to update a pointer using at least one previous value of the one or more configuration values.

[0034] In some embodiments, the method includes copying the new values ​​into a portion of one or more registers followed by causing the new values ​​to replace previous values ​​for use by the transmit engine.

[0035] In some embodiments, at least one of the one or more configuration values ​​includes at least one of a stride size value that defines the separation in memory between two of the data items that are transmitted simultaneously and a repetition value that defines the number of times a stride of a particular size is applied before applying a stride of a different size.

[0036] In some embodiments, the memory is configured to store a compiled set of code including one or more instances of transmit instructions and one or more instances of at least one instruction for updating a variable stride pattern, the method including executing each of the one or more instances of the transmit instructions for an allocated clock cycle that is a predetermined number of processor clock cycles after receiving a synchronization signal, and executing each of the one or more instances of the at least one instruction for updating a variable stride pattern for an allocated clock cycle that is the predetermined number of processor clock cycles after receiving the synchronization signal.

[0037] In some embodiments, the method includes, in response to execution of one or more additional instances of the transmit instruction by the execution unit, rearranging the multiple items of further data in the memory by transmitting the multiple items of further data via a loopback path back to be stored in the memory in an order determined by a current variable stride pattern active for the transmit engine.

[0038] According to a fifth aspect, there is provided a method for storing data received at a processing device, the method comprising: storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which a memory is accessed to store data received at the processing device; and, in response to receiving a plurality of data items at the processing device, storing the plurality of data items at locations in the memory identified by the pointer, and updating the pointer in accordance with the variable stride pattern for each reception and storage of one of the plurality of data items. The method further comprises executing at least one instruction for updating the variable stride pattern, wherein updating the variable stride pattern comprises updating at least one of the one or more configuration values ​​such that the new value of the at least one of the one or more configuration values ​​is used to store one or more further data items received at the processing device.

[0039] In some embodiments, the at least one instruction includes an instruction having a combined send and update function, and the method includes executing the instruction having the combined send and update function to both provide at least one new value of one or more of the configuration values ​​to the receiving engine for use by the receiving engine in storing one or more further data items, and to transmit the output data item from the memory.

[0040] In some embodiments, the at least one instruction, when executed by the execution unit, causes the receiving engine to copy at least one new value of the one or more configuration values ​​into a portion of one or more registers while continuing to update the pointer with the at least one previous value of the one or more configuration values.

[0041] In some embodiments, the method includes copying the new values ​​into a portion of one or more registers followed by causing the new values ​​to replace previous values ​​for use by the receive engine.

[0042] In some embodiments, at least one of the one or more configuration values ​​includes at least one of a stride size value that defines a separation in memory between locations in memory where data items are stored consecutively by the receiving engine and a repetition value that defines the number of times a stride of a particular size is applied by the receiving engine before applying a stride of a different size.

[0043] In some embodiments, the method includes, in response to receiving a plurality of items of further data via a loopback path of the processing unit, rearranging the plurality of items of further data in memory by storing the plurality of items of further data in an arrangement that is dependent on a current variable stride pattern active for the receive engine.

[0044] In some embodiments, the memory is configured to store a compiled code set including one or more instances of at least one instruction for updating a variable stride pattern, and the method includes executing the compiled code set such that each of the one or more instances of the at least one instruction is executed during an allotted clock cycle, the allotted clock cycle being a predetermined number of processor clock cycles after receipt of a synchronization signal at the processing device.

[0045] In some embodiments, at least one of the one or more configuration values ​​includes multiple configuration values.

[0046] In some embodiments, the method includes, upon execution of at least one instruction, updating a string of bits held in a configuration register of the processing device, the string of bits held in the configuration register being divided into a plurality of subsets of bits, each subset of bits mapping to one of a plurality of configuration values, whereby in response to writing the string of bits to the configuration register, the processing circuitry is configured to use the updated value for those of the plurality of configuration values ​​to which the subset is mapped.

[0047] In some embodiments, each of the multiple subsets of bits is shorter in length than its associated one of the multiple configuration values ​​to which it is mapped.

[0048] In some embodiments, the plurality of configuration values ​​includes a plurality of stride size values.

[0049] In some embodiments, the plurality of configuration values ​​includes a plurality of repetition values, each of which is associated with one of the stride size values ​​and specifies the number of times that the associated one of the stride size values ​​is applied before the processing circuitry applies a different one of the stride size values.

[0050] In some embodiments, the one or more registers are control and status registers that belong to a thread executing on an execution unit.

[0051] According to a sixth aspect, there is provided a computer program comprising computer readable instructions which, when executed by at least one processor, cause the computer program to perform a method according to the fourth or fifth aspect and any embodiment thereof.

[0052] According to a seventh aspect, there is provided a non-transitory computer readable medium comprising a computer program according to the sixth aspect.

[0053] For a better understanding of the present invention, reference will now be made, by way of example, to the accompanying drawings, which show how the same may be carried into effect. [Brief explanation of the drawings]

[0054] [Figure 1] FIG. 2 is a schematic block diagram of a processing device. [Figure 2] FIG. 1 is a schematic block diagram of multiple thread contexts. [Figure 3] FIG. 2 is a further schematic block diagram of a processing device. [Figure 4] FIG. 1 is a diagram of a single stride pattern. [Figure 5]FIG. 1 is a diagram of a dual stride pattern. [Figure 6] FIG. 1 is a diagram of a triple stride pattern. [Figure 7] FIG. 10 is a diagram of a stride pattern including a negative stride. [Figure 8] FIG. 2 is a diagram of a transmit / receive engine that is part of the processing unit. [Figure 9A] FIG. 10 is a diagram of fields of a first example transmission command. [Figure 9B] FIG. 10 is a diagram of fields of a second example transmission command. [Figure 10] FIG. 10 is a diagram of fields of a third example send instruction that combines send and put functions. [Figure 11A] FIG. 1 is a schematic diagram of a processing unit including multiple tiles. [Figure 11B] FIG. 1 is a schematic diagram of multiple tiles alternating between calculation and exchange phases. [Figure 11C] 1 is a schematic diagram of a BSP supercycle of computation and exchange, including barrier synchronization. [Figure 12] FIG. 1 is a diagram of compiling codelets for execution on tiles of processing units. [Figure 13] 1 is a flowchart of a method for transmitting data from a processing device. [Figure 14] 1 is a flow chart of a method for receiving data from a processing device. [Figure 15] FIG. 10 is a further schematic block diagram of a processing device showing a loopback path. [Figure 16] 1 illustrates a matrix transpose operation that can be performed using a variable stride function. [Figure 17] 10 shows the transmit pattern applied by the transmit engine to perform a matrix transpose operation when loopback mode is active. [Figure 18] 10 shows the stored pattern used by the receive engine to perform a matrix transpose operation when loopback mode is active. [Figure 19]1 is a flowchart of a method for rearranging data by sending the data over a loopback path. [Figure 20] 1 illustrates two fields of a trigger register according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0055] FIG. 1 illustrates an example processing device 40 according to an embodiment of the present disclosure. The processing device 40 includes a multithreaded processing unit 10 in the form of a barrel thread processing unit and a local memory 11 (i.e., on the same tile in the case of a multi-tile array or on the same chip in the case of a single processor chip). A barrel thread processing unit is a type of multithreaded processing unit in which pipeline execution time is divided into a repeating sequence of interleaved time slots, each of which may be owned by a given thread, as will be discussed in more detail shortly. The memory 11 includes an instruction memory 12 and a data memory 22 (which may be implemented by different addressable memory units or different regions of the same addressable memory unit). The instruction memory 12 stores machine code executed by the processing unit 10, while the data memory 22 stores both data operated on by the executed code and data output by the executed code (e.g., as a result of such operations).

[0056] The memory 12 stores various different threads of the program, each thread containing a sequence of individual instructions for performing a particular task. Note that an instruction referred to herein means a machine code instruction, i.e., an instance of a basic instruction of the processing unit's instruction set, consisting of a single opcode and zero or more operands.

[0057] The programs stored in memory 12 include a supervisor subprogram that may be structured as multiple worker threads and one or more supervisor threads. In an embodiment, some or all of the worker threads each take the form of an individual "codelet."

[0058] Within processing unit 10, multiple different ones of the threads from instruction memory 12 may be interleaved by a single execution pipeline 13 (although typically only a subset of all threads stored in instruction memory may be interleaved at any given time throughout the program). Multithreaded processing unit 10 includes multiple context register files 26, each configured to represent the state (context) of a different one of the simultaneously executing threads, a shared execution pipeline 13 common to the simultaneously executing threads, and a scheduler 24 for scheduling the concurrent threads for execution by the shared pipeline in an interleaved, preferably round-robin, manner. Processing unit 10 is connected to a shared instruction memory 12 common to the multiple threads and a shared data memory 22 also common to the multiple threads.

[0059] Execution pipeline 13 includes a fetch stage 14, a decode stage 16, and an execute stage 18 that includes execution units capable of performing arithmetic and logical operations, address calculations, load and store operations, and other operations defined by the instruction set architecture. Each of the context register files 26 includes an individual set of registers for representing the program state of an individual thread.

[0060] An example of the registers that make up each of the context register files 26 is shown schematically in Figure 2. Each of the context register files 26 includes one or more individual control registers 28, which in turn include at least a program counter (PC) for each thread (for tracking the instruction address at which the thread is currently executing) and, in embodiments, a set of one or more status registers (SRs) that record the current state of each thread (whether it is currently executing or whether it has paused, for example, because it has encountered an error). Each of the context register files 26 also includes a set of individual operand registers (OP) 32 for temporarily holding operands of instructions executed by each of the threads, i.e., values ​​that are operated on by or result from operations defined by the opcodes of the instructions of each of the threads at execution time. It will be understood that each of the context register files 26 may optionally include one or more individual registers of other types (not shown). It should also be noted that while the term "register file" may be used to refer to a group of registers within a common address space, this is not necessarily the case in this disclosure, and each hardware context 26 (each register set 26 representing each context) may more generally include one or more such register files.

[0061] In an embodiment, the configuration includes one worker context register file CX0...CX(M-1) for each of the number M of simultaneously executable threads (M=3 in the illustrated example, but not limited to this), and one additional supervisor context register file CXS. The worker context register files are reserved for storing the context of the worker threads, and the supervisor context register file is reserved for storing the context of the supervisor thread. Note that in an embodiment, the supervisor context is special in that it has a different number of registers than each of the workers. Each of the worker contexts preferably has the same number of status registers and operand registers as each other. In an embodiment, the supervisor context may have fewer operand registers than each of the workers. Examples of operand registers that a worker context may have and the supervisor does not include include floating-point registers, accumulate registers, and / or dedicated weight registers (for holding neural network weights). In an embodiment, the supervisor may also have a different number of status registers. Furthermore, in embodiments, the instruction set architecture of processing unit 40 may be configured such that worker threads and supervisor threads execute some different instruction types, but also share some instruction types.

[0062] Fetch stage 14 is connected to fetch instructions for execution from instruction memory 12 under the control of scheduler 24. Scheduler 24 is configured to control fetch stage 14 to fetch instructions from each of a set of concurrently executing threads in turn in a repeating sequence of time slots, thereby dividing the resources of pipeline 13 into multiple time-interleaved time slots. For example, the scheduling scheme may be round robin or weighted round robin. Another term for a processor operating in this manner is a barrel-threaded processor.

[0063] In some embodiments, scheduler 24 has access to one of each thread's status registers, SR, which indicates whether the thread is paused, and thereby scheduler 24 actually controls fetch stage 14 to fetch instructions for only the currently active thread. In embodiments, preferably, each time slot (and corresponding context register file) is always owned by one thread or another, i.e., each slot is always occupied by some thread and each slot is always included in scheduler 24's sequence, although it is possible that the thread occupying any given slot happens to be paused at the time, in which case instruction fetching for the respective thread is postponed when the sequence comes around to that slot. Alternatively, for example, in a less preferred alternative implementation, it is not excluded that some slots may be temporarily free and removed from the scheduled sequence. When referring to the number of time slots that an execution unit is operable to interleave, etc., this refers to the maximum number of slots that the execution unit can execute simultaneously, i.e., the number of simultaneous slots that the execution unit's hardware supports.

[0064] The fetch stage 14 has access to a program counter (PC) for each of the contexts. For each thread, the fetch stage 14 fetches the next instruction for that thread from the next address in program memory 12 indicated by the program counter. The program counter increments each execution cycle unless a branch instruction takes place. The fetch stage 14 then passes the fetched instruction to the decode stage 16 to be decoded, which then passes an indication of the decoded instruction, along with the decoded addresses of any operand registers 32 specified in the instruction, to the execution units 18 to be executed. The execution units 18 have access to the operand registers 32 and control registers 28, which the execution units 18 can use when executing instructions based on the decoded register addresses, such as in the case of arithmetic instructions (e.g., by adding, multiplying, subtracting, or dividing the values ​​in two operand registers and outputting the result to another operand register for the respective thread). Alternatively, if the instruction defines a memory access (load or store), the load / store logic of the execution unit 18 loads a value from the data memory into the respective thread's operand register or stores a value from the respective thread's operand register into the data memory 22 according to the instruction. Alternatively, if the instruction defines a branch or status change, the execution unit changes the value of one of the program counter PC or status register SR accordingly. Note that while an instruction of one thread is being executed by the execution unit 18, an instruction from the thread in the next time slot in the interleaved sequence may be decoded by the decode stage 16, and / or while an instruction is being decoded by the decode stage 16, an instruction from the thread in the following time slot thereafter may be fetched by the fetch stage 14 (however, in general, the scope of the present disclosure is not limited to one instruction per time slot; for example, in alternative scenarios, a batch of two or more instructions per time slot may be issued from a given thread).In this way, interleaving advantageously hides latencies within the pipeline 13 in accordance with known barrel threading techniques.

[0065] Reference is now made to Figure 3, which shows a further diagram of processing unit 40. Processing unit 40, as previously described, includes execution unit 18 and data memory 22. Figure 3 also shows transmit engine 300 and receive engine 320 of processing unit 40. Transmit engine 300 and receive engine 320 are configured to implement transmit and receive operations according to stride patterns stored as part of a compiled code set held in instruction memory 12.

[0066] The transmit engine 300 is configured to transmit data items from memory 22 under the control of the execution unit 18. The transmit engine 300 is responsive to SEND instructions executed by the execution unit 18, which cause the transmit engine 300 to transmit data items from memory 22 to a destination external to the processing unit 40. The transmit engine 300 maintains a read pointer that identifies the location in memory 22 of the next data item to be transmitted. The transmit engine 300 updates the read pointer between the transmission of each data item. In response to execution of a SEND instruction by the execution unit 18, the transmit engine 300 causes one or more data items to be transmitted on the data output interface 310 of the processing unit 40. Each of these data items transmitted in response to execution of a SEND instruction is identified by a read pointer, which is updated between the transmission of each item. For example, when a SEND instruction is executed by execution unit 18 to transmit two data items, transmit engine 300 accesses the first of these items at a location in memory 22 identified by the read pointer and transmits this first data item on data output interface 310. Following transmission of the first data item, transmit engine 300 updates the read pointer to point to the second data item. Transmit engine 300 then causes the second data item to be transmitted (on data output interface 310) from the new location in memory 22 identified by the read pointer.

[0067] The read pointer updates between data item transmissions are performed according to a variable stride pattern stored within the transmit engine 300. The variable stride pattern stored by the transmit engine 300 is referred to herein as the output stride pattern. The output stride pattern includes one or more configuration values ​​referred to herein as output configuration values. The output configuration values ​​specify the stride behavior of the transmit engine 300 (i.e., how the read pointer is updated). The output stride pattern is configurable by the execution unit 18. The execution unit 18 is configured to execute one or more instructions to update the output configuration value representing the output stride pattern used by the transmit engine 300. The execution unit 18 can update the configuration value such that transmission of a first set of data items performed by the transmit engine 300 before the update is performed using a first variable stride pattern represented by a first set of one or more configuration values, while transmission of a second set of data items performed by the transmit engine 300 after the update is performed using a second variable stride pattern represented by a second set of one or more configuration values.

[0068] Thus, execution unit 18 can set a variable stride pattern when the data sets to be transmitted are arranged in a regular repeating pattern in memory 22. In this way, execution unit 18 can transmit a data set without having to specify an address each time an item of data that is arranged non-contiguously with respect to a previously transmitted data item is transmitted by transmit engine 300. This allows for a reduction in the code size stored in instruction memory 12.

[0069] In an embodiment, execution unit 18 is configured to execute various types of SEND instructions to cause data to be transmitted by transmit engine 300 .

[0070] Referring to FIG. 9A , an example of fields of a first type SEND instruction 900 that may be executed by execution 18 to cause a transmission is shown. SEND instruction 900 includes an opcode 910 that identifies instruction 900 as a first type SEND instruction. SEND instruction 900 specifies a number 920 of data items to be transmitted from memory 22 upon execution of the instruction. SEND instruction 900 includes an address field 930 that allows specification of an address within memory 22 from which data is to be transmitted. Executing instruction 900 causes the number of data items specified by field 920 to be transmitted, starting from the address specified by field 930. Specifically, the first data item transmitted in response to the instruction is transmitted from the address specified by field 930, and the remaining data items are transmitted from addresses determined by applying an output stride pattern, starting from the address specified by field 930. Field 940 is an optional field that specifies one or more directions in which data is transmitted from processing unit 40. This field 940 may be referred to as the East / West indicator and is described in more detail in US Patent Application Publication No. 15 / 886,315, which is incorporated by reference.

[0071] 9B, an example of the fields of a second type SEND instruction 900b that may be executed by execution unit 18 to cause a transmission is shown. SEND instruction 900a includes an opcode 910a that identifies instruction 900a as a second type SEND instruction. Like SEND instruction 900, SEND instruction 900a includes a field 920 that specifies the number of data items to be transmitted in response to execution of the instruction and a field 940 that specifies one or more directions in which the data will be transmitted from processing unit 40. Unlike first type SEND instruction 900, second type SEND instruction 900a does not include an address field. When an instance of second type SEND instruction 900a is executed by execution unit 18, one or more data items are transmitted starting from an address identified by the current state of a read pointer maintained by transmit engine 300. This current state of the read pointer reflects the state of the read pointer after the transmission of the previous data item by transmit engine 300.

[0072] In this manner, the execution unit 18 may execute a SEND instruction to send data from a specified address, or may execute a SEND instruction to send data based on the current state of the transmit engine 300.

[0073] Referring again to FIG. 3 , it is shown that the receive engine 320 may be implemented as part of the data input interface 330. The receive engine 320 is configured to store in memory 22 data items received at the data input interface 330 from sources external to the processing unit 40 (e.g., other processing units). The receive engine 320 maintains a write pointer that identifies the location in memory 22 where the next data item received on the interface 330 will be stored. The write pointer may be updated at any time by the execution unit 18 executing a memory pointer update instruction, which writes a new value of the write pointer to the receive engine 320. Additionally, independent of this intervention by the execution unit 18, the receive engine 320 updates the write pointer between receipt of each data item by the receive engine 320. In response to receipt of a data item at the processing unit 40, the receive engine 320 stores the data item at the location in memory 22 identified by the write pointer. Following storage of the data item in memory 22, the receive engine 320 updates the write pointer. The updated write pointer points to a further location in memory 22 where a further data item will be stored upon receipt of the further data item at processing unit 40 .

[0074] The write pointer updates are performed according to a variable stride pattern stored in the receive engine 320. The variable stride pattern stored by the receive engine 320 is different from the variable stride pattern used by the transmit engine 300. The variable stride pattern stored by the receive engine 320 is referred to herein as an input stride pattern. Furthermore, one or more configuration values ​​that define the input stride pattern may be referred to as input configuration values.

[0075] The input configuration values ​​specify the stride behavior (i.e., how the write pointer is updated) of receive engine 320 while storing different received data items in memory 22. The input stride pattern is configurable by execution unit 18 by modifying the input configuration values. Execution unit 18 can update the input configuration values ​​such that a first set of data items received at processing unit 40 before the update are stored in locations in memory 22 determined according to a first input stride pattern represented by a first set of input configuration values, while a second set of data items received at processing unit 40 after the update are stored in locations in memory 22 determined using a second input stride pattern represented by a second set of input configuration values.

[0076] Thus, execution unit 18 can set a variable stride pattern for receiving data, which is useful when placing data sets in memory 22 in a regular, repeating pattern. In this way, execution unit 18 can store received data sets in appropriate locations in memory 22 (e.g., locations that facilitate efficient subsequent data processing) without the need for execution unit 18 to intervene to update a write pointer each time data is written to a new location in memory 22 that is non-contiguous with respect to the previous write location. This allows for a reduction in the size of the code stored in instruction memory 12.

[0077] In an example embodiment, one or more configuration values ​​maintained by the transmit engine 300 and / or the receive engine 320 may define a stride pattern that includes one or more loops. Examples of such loops are shown in FIGS. 4-7 and discussed in more detail below. When accessing memory 22 according to such a looped stride pattern, the engine 300 / 320 performs multiple accesses strided by a first stride value defined by the respective configuration values. Following this, the engine 300 / 320 modifies the address for the next access by a second stride value, and then reapplies the first stride value to stride between each of the further accesses. Table 1 shows an example of configuration values ​​that may be stored within the transmit engine 300 and the receive engine 320 to define such a stride pattern. In the table, each of these stride values ​​is represented by the name of the register in which the respective stride value is stored.

[0078] [Table 1]

[0079] The output configuration registers are maintained within the transmit engine 300 and store one or more configuration values ​​(i.e., output configuration values) that define the stride pattern used by the transmit engine 300. The output configuration values ​​include a stride size value that defines the size of each stride and a repeat value that defines the number of times each stride is applied before a different stride of an outer loop (in which the current stride loop is nested) is applied (if such an outer loop is defined) or until the stride pattern is repeated. The stride size can be positive or negative to indicate whether the stride is a memory forward stride to a higher memory address (if the stride is positive) or a memory backward stride to a lower memory address (if the stride is negative). Table 1 shows configuration values ​​for three different types of strides. The first stride is performed in the inner loop and is referred to herein as the inner stride. The second stride is performed in the loop in which the inner loop is nested and is referred to herein as the mezzostride. The third stride is performed in the loop in which the mezzostride loop is nested and is referred to herein as the outer stride.

[0080] The first repetition value (referred to herein as the inner repetition value) held in $OUTGOING_RPT0 defines the number of times the inner stride is applied by the transmit engine 300 before a mezzo stride is performed or the entire variable stride pattern is repeated. In an embodiment, the size of the inner stride is not configurable but is hard-coded to the smallest unit of memory access (i.e., the size of one atom of memory 22). The second stride size value (referred to herein as the mezo size) held in $OUTGOING_STR0 defines the size of the mezzo stride that is applied after the inner stride has been applied the number of times defined in the $OUTGOING_RPT0 register. The second stride repetition value (referred to herein as the mezo repetition value) held in $OUTGOING_RPT1 defines the number of times the mezzo stride is applied by the transmit engine 300 before an outer stride is applied or the entire variable stride pattern is repeated. The third stride size value (referred to herein as the outer size) held in $OUTGOING_STR1 defines the size of the outer stride that is applied after the mezostride has been applied the number of times defined in the $OUTGOING_RPT1 register. The outer stride is applied only once per stride pattern repetition.

[0081] The input configuration registers are registers maintained within the receive engine 320 that store one or more configuration values ​​(i.e., input configuration values) that define the stride pattern used by the receive engine 320. These values ​​have equivalent meanings to the output configuration values ​​used by the transmit engine 300. The first repetition value (referred to herein as the inner repetition value) maintained within $INCOMING_RPT0 defines the number of times an inner stride is applied by the receive engine 320 before a mezzostride is performed or the stride pattern is repeated. In an embodiment, the size of the inner stride is not configurable but is hard-coded to the value of the smallest unit of memory access (i.e., the size of one atom of memory 22). The second stride size value (referred to herein as the mezzostride size) maintained within $INCOMING_STR0 defines the size of the mezzostride that is applied after the inner stride has been applied the number of times defined in the $INCOMING_RPT0 register. A second stride repeat (herein referred to as the meso repeat value) value held in $INCOMING_RPT1 defines the number of times a meso stride is applied by Receive Engine 320 before an outer stride is applied or the stride pattern is repeated. A third stride size value (herein referred to as the outer stride size) held in $INCOMING_STR1 defines the size of the outer stride that is applied after a meso stride has been applied the number of times defined in the $INCOMING_RPT1 register. An outer stride is applied only once per stride pattern repetition.

[0082] Referring to Figure 4, which shows an example of a stride pattern, this stride pattern may be applied by transmit engine 300 to identify locations in memory 22 from which data is transmitted externally to a processing unit, or may be applied by receive engine 320 to select locations in memory 22 where data is stored. Figure 4 shows a portion 400 of memory 22 divided into units (called atoms) that represent storage space for storing a single data item to be transmitted from or received by processing unit 40 in a single data packet. In an embodiment, the size of each of these atoms is 32 bits. The hashed units of memory 22 are what are accessed by transmit engine 300 or receive engine 320 to transmit and receive data.

[0083] The strides shown in FIG. 4 are all of a single stride size (e.g., +3 units in this example). These single-stride-size strides can be executed using mesostrides or outer strides as discussed above with respect to Table 1. In this example, inner strides are not used by engine 300 / 320, which can be achieved by setting the inner repetition value (e.g., held in $OUTGOING_RPT0 or $INCOMING_RPT0) to zero so that the inner stride is not executed. Similarly, either mesostrides or outer strides not used by engine 300 / 320 can be configured to be unused by setting the size value or repetition value of the respective stride type to zero. Either mesostride or outer stride (used by engine 300 / 320) is applied by setting the stride size value for that stride to the value of the +3 atom in memory 22.

[0084] 5, which illustrates a further example of a stride pattern, this stride pattern may be applied by transmit engine 300 to select data from memory 22 to be transmitted externally to processing unit 40, or by receive engine 320 to select locations within memory 22 where data is stored. Similar to FIG. 4, portion 500 of memory 22 is shown divided into (e.g., 32-bit) atoms for storing respective data items to be transmitted / received within a single data packet. The hashed boxes represent atoms accessed by engine 300 / 320 to transmit data from or store data in.

[0085] The strides shown in FIG. 5 include two different sizes of strides. One of these strides involves striding within memory 22 by +2 units between accesses to memory 22. The other of these strides involves striding within memory 22 by +5 units between accesses to memory 22. Such a pattern can be achieved by setting the inner repeat value (e.g., held in $OUTGOING_RPT0 or $INCOMING_RPT0) to zero so that the inner stride is not executed. The mezo stride size is set to +2, while the mezo repeat value is set to 2. As a result, as shown in FIG. 5, a stride of size +2 is executed after the first access to memory 22, followed by a stride of size +2 after the second access to memory 22. The outer stride size is set to +5. As a result, and as shown in FIG. 5, when the mezo stride is executed the number of times specified by the outer repeat value, the outer stride is executed once.

[0086] 6, which illustrates a further example of a stride pattern that may be applied by transmit engine 300 to select data from memory 22 to be transmitted externally to processing unit 40, or that may be applied by receive engine 320 to select a location within memory 22 where data is stored. FIG. 6 illustrates the use of an example stride pattern to perform an access to a portion 600 of memory 22.

[0087] The strides shown in FIG. 6 include strides of three different sizes (i.e., +1, +3, and +5). As part of implementing the stride pattern shown in FIG. 6, the inner repeat value (e.g., held in the $INCOMING_RPT0 and $OUTGOING_RPT0 registers) is set to 2. As a result, a stride of size +1 is executed after the first access to memory 22, followed by another stride of size +1 after the second access to memory 22, as shown in FIG. 6. The meso stride size is set to +3, and the meso repeat value is set to 2. As a result, a +3 stride is executed after completing the set of inner strides, as shown in FIG. 6. The +3 stride is executed twice before executing the outer stride. The outer stride size is set to +5, resulting in a +5 stride being executed after all of the meso strides and inner strides specified by the inner repeat and meso repeat values ​​of the pattern have been executed. After the outer stride, the same pattern of inner strides and meso strides is repeated again.

[0088] 7, which illustrates a further example of a stride pattern that may be applied by transmit engine 300 to select data from memory 22 to be transmitted externally to processing unit 40, or that may be applied by receive engine 320 to select a location within memory 22 where data is stored. FIG. 7 illustrates an example of this stride pattern for performing an access to a portion 700 of memory 22.

[0089] In the pattern of Figure 7, the outer stride is set to a negative value (-14), which causes the engine 300 / 320 pointer to jump back to a lower memory address after completing a set of mezzo strides. The mezzostride size is set to +5, and the mezo repeat value is 3. As a result, after completing three mezzo strides, the pointer points to an atom that is 15 atoms away from the first atom accessed. By striding -14 atoms, the engine 300 / 320 next accesses the atom immediately following the first atom accessed. The engine 300 / 320 then repeats the set of mezzo strides before striding backward again with the outer stride.

[0090] In Figure 7, a first set of mezzostrides and a first outer stride are shown in the top half of the figure, while a second set of mezzostrides and a second outer stride are shown in the bottom half of the figure. The stride pattern shown in Figure 7 will cause access to all of the atoms of portion 700 of memory 22 if a sufficient number of data items are transmitted or received using this stride pattern. However, to simplify the illustration, only a subset of the atoms of portion 700 of memory 22 is shown as being accessed.

[0091] 8, there is shown an example of an engine 300 / 320, which may be the receive engine 320 or the transmit engine 300. Unless explicitly stated as applying to the receive engine 320 or the transmit engine 300, features described with respect to FIG.

[0092] Engine 300 / 320 includes processing circuitry 800 for accessing memory 22 according to the described variable stride pattern. Operations described herein as being performed by transmit engine 300 or receive engine 320 are performed by that engine's processing circuitry 800. Processing circuitry 800 may be a processor configured to execute computer-readable instructions to perform the described operations. Additionally or alternatively, processing circuitry 800 may include dedicated hardware (e.g., a programmable gate array or an application-specific integrated circuit) for performing the described operations.

[0093] If belonging to transmit engine 300, processing circuit 800, in response to SEND instructions executed by execution unit 18, loads data items from memory 22 at locations determined according to a defined variable stride pattern and dispatches them in data packets. Specifically, in response to execution of each such SEND instruction by execution unit 18, transmit engine 300 causes the data item to be accessed from a location in memory 22 identified by a pointer in transmit engine 300 and transmitted in a data packet from processing unit 40. Processing circuit 800 then increments the pointer by an amount specified by the variable stride pattern of transmit engine 300 to point to the next data item in memory 22 to be transmitted by transmit engine 300. If the executed SEND instruction specifies that multiple data items are to be transmitted, transmit engine 300 transmits the next data item pointed to by the pointer without waiting for another instance of the SEND instruction to be executed by execution unit 18. However, if the SEND instruction did not specify multiple data items, the next data item pointed to by the pointer will only be sent if a further instance of the SEND instruction is executed by execution unit 18.

[0094] If processing circuit 800 belongs to receive engine 320, circuit 800 is configured to, upon receiving a data item, store the data item at a location in memory 22 identified by a pointer in receive engine 320. Processing circuit 800 then increments the pointer by an amount specified by the variable stride pattern of receive engine 320 to point to the next location in memory 22 where data is stored as further data items are received.

[0095] A pointer can include two parts: a base address and a delta. The base address is stored in storage 810a, while the delta is stored in storage 810b. The base address and delta together represent a pointer, which identifies the next location in memory 22 to be accessed by the engine 300 / 320 to obtain data for transmission (in the case of the transmit engine 300) or to store received data (in the case of the receive engine 320). The address identified by the pointer is given by the sum of the base address and the delta. The base address is a semi-static value and is not updated directly by a SEND instruction or by the stride activity of the engine 300 / 320. The delta, on the other hand, can be set by execution of a SEND instruction (in the case of the transmit engine 300) or a memory pointer update instruction (in the case of the receive engine 320). The delta value is also updated by the engine 300 / 320 according to the variable stride pattern currently active for that engine 300 / 320. For example, after performing the first access to memory 22 following the establishment of the stride pattern, the delta value is incremented by the stride value specified as part of the active stride pattern for the respective engine 300 / 320. A second access to memory 22 is then made at the newly identified location. Processing circuit 800 continues to update the delta according to the active stride pattern between each access to memory 22. In this manner, the pointer is updated between each access to memory 22 while the stride pattern is active.

[0096] Engine 300 / 320 includes several registers that control the operation of engine 300 / 320. These registers are part of the supervisor thread's control registers 28 discussed above with respect to Figure 2. Control registers 28 are accessible to the supervisor thread executing on execution unit 18.

[0097] Counter registers 820a and 820b are provided as part of engine 300 / 320. These counters allow engine 300 / 320 to determine its progress through its configured stride pattern and therefore the next stride to be applied. Counter registers 820a and 820b are initialized to zero at the beginning of a stride pattern application. A value held in a first counter register 820a (designated STRPGS0) is incremented each time an inner stride is executed. When the value of counter register 820a reaches the inner stride repeat value, another stride (i.e., a mezzo stride or an outer stride) is executed instead of the inner stride, and counter register 820a is reset. A value in a second counter register 820b (designated STRPRGS1) is incremented each time a mezzo stride is executed. When the value of counter register 820b reaches the mezzo repeat value, an outer stride is executed instead of a mezzo stride, or the stride pattern is repeated (if an outer stride is not configured as part of the stride pattern). In addition, the second counter register 820b is reset.

[0098] Counter registers 820a, 820b are read-only status registers of the supervisor thread. Registers 820a, 820b are read-only in the sense that the supervisor thread cannot modify their values, although the values ​​held in registers 820a, 820b can be modified (i.e., by incrementing or resetting) by the processing circuitry 800 of engine 300 / 320. Registers 820a and 820b of transmit engine 300 can be referred to as $OUTGOING_STRPGS0 and $OUTGOING_STRPGS1, respectively. Registers 820a and 820b of receive engine 320 can be referred to as $INCOMING_STRPGS0 and $INCOMING_STRPGS1, respectively.

[0099] The engine 300 / 320 includes a configuration register 830 that stores one or more stride configuration values.

[0100] Each of the registers 830 is divided into two portions. The first portion of each register contains configuration values ​​for the currently active variable stride pattern. The portion of the register 830 that holds the configuration values ​​defining the currently active stride pattern is denoted by ".NOW." The processing circuit 800 uses these configuration values, held in the .NOW portion of the register 830, to update the read / write pointers between different accesses to the memory 22.

[0101] In addition to storing the currently active configuration values, configuration register 830 also stores a set of additional configuration values ​​that define additional stride patterns to be applied subsequent to the currently active stride pattern. The set of additional configuration values ​​is indicated by “.NXT.” In an embodiment, to update the currently active stride pattern, execution unit 18 first executes one or more instructions to update one or more of the configuration values ​​held in the .NXT portion of register 830. Execution unit 18 then causes the .NXT value to be copied to the .NOW portion of register 830 to overwrite the .NOW value after updating the .NXT value. Writing the value to the .NOW portion of register 830 commits these values. Writing the .NOW value is performed by execution unit 18 writing a trigger value to trigger register 840, which is part of engine 300 / 320. In addition to copying the .NXT value to overwrite the .NOW value, writing to trigger register 840 also causes the values ​​of counters 820a, 820b to be reset.

[0102] Each of the configuration registers 830 stores two values, one of which is a .NXT value and the other is a .NOW value. The configuration registers 830 include a register (RPT0) that stores two inner stride repeat values, a register (STR0) that stores two mezzostride size values, a register (RPT1) that stores two mezzostride repeat values, and a register (STR1) that stores two outer stride size values. In the transmit engine 300, the registers 830 contain two sets of output configuration values ​​described above with respect to Table 1. In the receive engine 320, the registers 830 contain two sets of input configuration values ​​described above with respect to Table 1.

[0103] Having the ability to store two sets of configuration values ​​(i.e., .NOW values ​​and .NXT values) that define two different stride patterns allows engine 300 / 320 to prepare the configuration values ​​for a new stride pattern while the old stride pattern is still being used to access memory 22. In this way, engine 300 / 320 can seamlessly move between stride patterns without interrupting current transmit and receive activity.

[0104] In the case of receive engine 320, one or more instructions for updating the stride configuration values ​​may be executed by execution unit 18 while one or more data packets are being received by processing unit 40. In response to execution of one or more instructions by execution unit 18, execution unit 18 causes one or more of the values ​​held in the .NXT configuration registers to be updated. Meanwhile, receive engine 320 continues to use the configuration values ​​held in the .NOW registers to update the write pointers to memory 22. Execution unit 18 then causes a trigger value to be written to register 840 to cause processing circuit 800 of receive engine 320 to commit those new values ​​by overwriting the current .NOW value with the new value written to the .NXT portion of register 830. These new values ​​are then used by receive engine 320 to update the pointers to determine the locations in memory 22 where additional data items will be stored. In this manner, receive engine 320 switches between using two different stride patterns.

[0105] The transmit engine 300 can perform a seamless shift between the two stride patterns using a new type of instruction, referred to herein as the SENDPSTREX instruction. The SENDPSTREX instruction represents a third type of SEND instruction, in addition to the SEND instruction 900 and SEND instruction 900a described above. The SENDPSTREX instruction combines send and put functions. When executed by the execution unit 18, this instruction performs a transmit from memory 22 in addition to updating configuration values ​​held in registers 830. Use of this instruction allows the stream of transmits from memory 22 to continue uninterrupted while simultaneously updating the configuration values ​​that define the stride pattern. When the SENDPSTREX instruction is executed by the execution unit 18, the execution unit 18 updates one or more of the configuration values ​​held in the .NXT portion of the configuration registers 830. As a result of executing this instruction, the execution unit 18 causes the transmit engine 300 to transmit a data item from memory 22 at a location identified by the read pointer of the transmit engine 300. After transmit engine 300 executes both the transmit instruction and the update of the value of .NXT, execution unit 18 causes the trigger value to be written to register 840. In response, transmit engine 300 commits the new configuration value by copying it from the .NXT portion of register 830 to the .NOW portion of register 830.

[0106] In addition to being used to update output stride configuration values, the SENDPSTREX instruction 1000 also allows for the updating of input configuration values ​​without interrupting a transmission sequence. For example, instruction memory 12 may store instructions for transmitting multiple data items from memory 22. During the period in which the multiple data items are being transmitted, execution unit 18 may be required to intervene to update the input stride pattern currently being used by receive engine 320. SENDPSTREX 1000 enables this to be accomplished by combining the functionality of a SEND instruction for transmitting data with the functionality of a PUT instruction for updating input configuration values. This greatly simplifies the compilation of programs held in instruction memory 12, as a sequence of SEND instructions can be compiled without relying on the data reception pattern at processing unit 40. Once the data reception pattern at processing unit 40 is determined, some of the SEND instructions for allocation to processing unit 40 can be converted to SENDPSTREX instructions (to update the input stride configuration as needed) without having to rearrange the execution timing of the SEND instructions.

[0107] 10, an example of the fields of a SENDPSTREX instruction 1000 is shown. A first field 1010 is an opcode that identifies the SENDPSTREX instruction 1000 as a SENDPSTREX instruction. A second field 1020 identifies which of the registers (i.e., $OUTGOING_RPT0, $OUTGOING_STR0, $OUTGOING_RPT1, $OUTGOING_STR1, $INCOMING_RPT0, $INCOMING_STR0, $INCOMING_RPT1, $INCOMING_STR1) should be updated in response to execution of the SENDPSTREX instruction 1000. A third field 1030 provides the configuration value to be written to the identified register. A fourth optional field 940 specifies the direction along the exchange that one or more data items transmitted in response to the instruction will be sent. In an embodiment, the fourth field 940 may be set to a particular value (e.g., both the East and West flags are set to 0) so that no transmission is performed when the SENDPSTREX instruction is executed. In this manner, the SENDPSTREX instruction may be used to update configuration values ​​that define the variable stride pattern of the transmit engine 300 without transmitting any data. In an embodiment, the SENDPSTREX instruction does not include a field that specifies the number of data items to be transmitted when the instruction is executed; rather, a single data item is transmitted in response to execution of a single instance of the instruction.

[0108] 8 shows an additional register 850. This register may be referred to herein as the strall register 850. When residing in the Transmit Engine 300, the register 850 is labeled $OUTGOING_STRALL, and when residing in the Receive Engine 320, the register 850 is labeled $INCOMING_STRALL.

[0109] The strall register 850 allows all of the configuration values ​​that define the variable stride pattern of the engine 300 / 320 to be written with a single write to the strall register 850. The strall register 850 may store a single value that can be thought of as including multiple subfields, each mapping to a different one of the configuration values. For example, the strall register 850 may contain a single 16-bit value that includes four 4-bit subfields, each mapping to a different one of the configuration values. A mapping function defines how each subfield maps to a configuration value. For example, the mapping function may give the corresponding configuration value of a subfield as a power of two of the value of the subfield.

[0110] Thus, strall register 850 provides a means by which execution unit 18 can update multiple configuration values ​​for the stride pattern with a single write to register 850, as opposed to multiple writes over several clock cycles of the processor. When execution unit 18 writes to this register 850, processing circuit 800 uses the value written to register 850 to update multiple configuration values ​​in register 830. In an embodiment, strall register 850 is mapped to the .NXT configuration value, allowing multiple of these to be updated with a single write to register 850.

[0111] When implemented in the transmit engine 300, the strall register 850 may be used with the SENDPSTREX instruction. In this case, the zimm4 field 1020 discussed above may be used to specify the strall register 850 rather than one of the configuration registers 830. The zimm16 field 1030 specifies the value to be written to the strall register 850 in response to execution of the SENDPSTREX instruction to cause the processing circuit 800 to update multiple configuration registers 830.

[0112] In some embodiments, engine 300 / 320 includes multiple such strall registers, each associated with a different mapping function for mapping the subfields held in the respective strall register to configuration registers 830.

[0113] In one embodiment, there are three strall registers within transmit engine 300, labeled as follows: $OUTGOING_STR_2D0 $OUTGOING_STR_2D1 $OUTGOING_STR_1D

[0114] Additionally, within the receive engine 320 there are three strall registers labeled as follows: $INCOMING_STR_2D0 $INCOMING_STR_2D1 $INCOMING_STR_1D

[0115] Writing to these registers follows a fixed mapping in the hardware (with different mappings applied to each of the three strall registers in engine 300 / 320) and results in the derived value being written to a subset of the $INCOMING_RPTn / STRn.NXT or $OUTGOING_RPTn / STRn.NXT registers.

[0116] It has been described that after writing the .NXT field of configuration register 830, execution unit 18 writes a value to trigger register 840 to commit the value written to the .NXT field (i.e., copy it into the .NOW field). Execution unit 18 may perform this write to trigger register 840 in response to various conditions. First, if execution unit 18 executes an instruction (e.g., a SENDPXTREX or PUT instruction) to write to a strall register (e.g., strall register 850), execution unit 18 writes to this strall register and then writes to trigger register 840 to commit the new .NXT field resulting from the write to the strall register. In this case, no additional instruction needs to be executed to cause the write to trigger register 840; the write to trigger register 840 is performed in response to the instruction that causes the write to strall register 850. Second, if execution unit 18 executes one or more instructions (e.g., a SENDPXTREX or PUT instruction) to directly write to one or more .NXT fields in register 830, trigger register 840 is not automatically written. In this case, execution unit 18 executes a further instruction (e.g., a SENDPXTREX or PUT instruction) to write to trigger register 840 to commit the value written to the .NXT field.

[0117] In some example embodiments, the configuration registers 830 discussed above are part of the supervisor thread's control registers 28. These registers 28 are both readable and writable by the supervisor thread. The strall register 850 may be a writable one of the supervisor thread's control registers 28. The trigger register 840 may be a writable one of the supervisor thread's control registers 28.

[0118] In some embodiments, trigger register 840 includes an additional field, referred to herein as the CLR field, for resetting the variable stride state. See FIG. 20, which shows an example of a trigger register 840 that includes two different fields. Execution unit 18 can independently write to either of these fields to perform different actions.

[0119] Trigger register 840 includes a first field (denoted as NXT). When execution unit 18 executes a write to the NXT field, the value in the .NXT field is committed (i.e., the value in the .NXT field is copied to the .NOW field).

[0120] Trigger register 840 also includes a second field (denoted as CLR). When execution unit 18 performs a write to the CLR field, a clear trigger event is performed that causes the value of .NOW in configuration register 830 to be reset to its default value. These default values ​​may be all zeros, in which case the read / write pointers are incremented by one atom after each send / receive. In this case, any stride pattern may be explicitly specified within the code executed therein; i.e., the address values ​​used by the SEND and memory pointer update instructions may be used to stride to different addresses. However, the clear trigger event does not cause the value of .NXT in configuration register 830 to be reset. Thus, execution unit 18 can resume the variable stride implemented by engine 300 / 320 by performing a write to the NXT field of trigger register 840.

[0121] One example of a type of data for which it may be useful to transmit and receive such data in a repeating pattern is a matrix. Such matrices may be useful for implementing certain types of machine learning models, such as convolutional neural networks. Elements of a given matrix may be stored in a regular arrangement in memory 22 to be more efficient in terms of required code storage space, so that execution unit 18 configures transmit engine 300 to transmit data according to a stride pattern that updates pointers to individual elements of the matrix rather than specifying addresses for each individual element. Similarly, on the receive side, processing unit 40 may receive matrices from several different sources. For example, one source may provide one column of the matrix, while another source provides another column of the matrix. In this case, execution unit 18 of device 40 may configure receive engine 320 with a stride pattern to store elements of a first column non-contiguously and spaced apart by a fixed stride, then stride back in memory 22 and store elements of a second column spaced apart by the same fixed stride. Such a stride pattern may be similar to that shown in FIG. 7 .

[0122] In some embodiments, processing unit 40 is a tile 4 of a multi-tile processor 2. See Figure 11A, which shows an example of such a multi-tile processor 2. Each such processing unit 2 is implemented on its own integrated circuit.

[0123] The processing unit 2 includes an array 6 of multiple processor tiles 4 and an interconnect 34 connecting the tiles 4. The processing unit 2 may be implemented singly as one of multiple dies packaged within the same IC package. The interconnect 34 may also be referred to herein as a “switching fabric” 34 because it allows the tiles 4 to exchange data with each other. Each tile 4 includes individual instances of execution units and memory. For example, a processing unit 2 may include on the order of hundreds or even thousands of tiles 4. For completeness, it should also be noted that references to an “array” herein do not necessarily imply a particular number of tiles 4 dimensions or physical layout.

[0124] In an embodiment, each processing unit 2 also includes one or more external links that allow the processing unit 2 to be connected to one or more other processing units (e.g., one or more other instances of the same processing unit 2). These external links may allow the processing unit 2 to be connected to a host system and to one or more other instances of processing unit 2 on the same IC package or card or on different cards. Processing units 2 receive work from the host in the form of application data that they process.

[0125] The interconnects 34 are configured to allow different tiles 4 in the array 6 to communicate with each other.

[0126] Further details of such a multi-tile processing unit 2 are provided in US Patent Application Publication No. 16 / 527410, which is incorporated by reference.

[0127] Referring to FIG. 11B , which shows an implementation of a BSP exchange scheme, in this scheme, each tile 4 executes a calculation stage 33 and an exchange stage 32 in alternating cycles, separated from each other by a barrier synchronization 30 between the tiles. In the example shown by FIGS. 11B and 11C , a barrier synchronization is placed between each calculation stage 33 and the subsequent exchange stage 32. During the calculation stage 33, each tile 4 executes one or more calculation tasks locally on the tile, but does not communicate any results of these calculations with any other tiles 4. In the exchange stage 32, each tile 4 is allowed to exchange one or more results of a calculation from a previous calculation stage with one or more other tiles 4, but does not perform any new calculation until it has received any data on which its task has a dependency from the other tiles 4. It does not send any data other than that calculated in the previous calculation stage to any other tiles 4. It is not excluded that other operations, such as operations related to internal control, may be executed within the exchange stage 32. Communications outside of the tile group can optionally utilize BSP mechanisms, but alternatively may not utilize BSP and instead use some other proprietary synchronization mechanism.

[0128] According to BSP principles, barrier synchronization 30 is placed at the seams where a calculation stage 33 transitions to an exchange stage 32, or where an exchange stage 32 transitions to a calculation stage 33, or both. That is, (a) all tiles 4 are required to complete their respective calculation stage 33 before any tile in the group is allowed to proceed to the next exchange stage 32, or (b) all tiles 4 in a group are required to complete their respective exchange stage 32 before any tile in the group is allowed to proceed to the next calculation stage 33, or (c) both of these conditions are enforced. In all three variants, it is the individual tiles that alternate between stages and the entire assembly that synchronizes. This sequence of exchange and calculation stages can then be repeated for multiple iterations. In BSP terminology, each iteration of the exchange and computation steps may be called a "superstep" (note, however, that this terminology is not always used consistently in the literature; individual exchange and computation steps may also be called supersteps individually, whereas in other cases the exchange and computation steps together are called a superstep, as is the terminology adopted herein).

[0129] It should also be noted that it is not excluded that multiple different independent groups of tiles 4 on the same processing unit 2 or different processing units 2 may each form separate individual BSP groups that operate asynchronously with one another, with the BSP cycle of calculation, synchronization, and exchange being imposed only within each given group, each group doing so independently of other groups. That is, a multi-tile array 6 may include multiple internally synchronized groups that each operate independently and asynchronously with respect to other such groups (discussed in more detail below). As discussed in more detail below, in some embodiments there is a hierarchical grouping of synchronization and exchange.

[0130] 11C shows (a) the BSP principle implemented between some or all groups 4i, 4ii, 4iii of tiles in array 6 when imposing a barrier synchronization (see above) from calculation stage 33 to exchange stage 32. Note that in this configuration some tiles 4 are allowed to start calculation 33 while other tiles 4 are still being exchanged.

[0131] The BSP model is used to exchange data between tiles 4 on a processing unit 2. Communication between tiles 4 of a processing unit 2 is performed in a time-deterministic manner, where data packets are transmitted without headers as in our prior application, U.S. Patent Application Publication No. 15 / 886,315. In addition, the BSP model can also be used to exchange data between processing units 2. Such exchange of data between processing units 2 is referred to as an external exchange 50'.

[0132] During the exchange phase, instructions executed by tiles are scheduled to execute at predetermined times to allow communication to occur in a time-deterministic manner. Instructions executed at predetermined times include SEND instructions and instructions used to update variable stride patterns (e.g., SENDPXTREX instructions). In this way, each tile 4 transmits each data item at the appropriate time, so that when the receiving tile 4 receives a data item, its write pointer points to the appropriate location in memory for storing that data item.

[0133] The execution timing of the SEND instruction and the instruction to update the variable stride pattern is aligned with a synchronization signal issued by the synchronization controller 36 shown in FIG. 11A. The synchronization controller 36 issues a signal to each tile 4 indicating the start of the exchange phase. The execution units 18 of the tile 4 then execute each instance of the SEND instruction and the variable stride pattern update instruction within an allocated processor clock cycle, each of which is a predetermined number of processor clock cycles after receipt of the synchronization signal at the respective tile 4.

[0134] In an embodiment of a multi-tile processing unit 2, each tile 4 includes an instance of the processing unit 40 discussed above. Each such tile 4 is configured to store and execute a compiled code set provided by a compiler. FIG. 12 is a schematic diagram illustrating the functionality of a compiler 70. The compiler receives such a graph 60 and compiles the functions in the graph into multiple codelets, which are contained in local programs labeled 72 in FIG. 12. Each local program is designed to be loaded into a particular tile of the computer. Each program includes one or more codelets 72a, 72b,... and a supervisor subprogram 73, each formed from a sequence of instructions. The compiler generates the programs to be linked together. Thus, if a compiled code set executing on one or more transmitting tiles 4 has a particular data pattern to send to a receiving tile 4, the linked compiled set executing on the receiving tile 4 configures the receiving engine 320 of that tile 4 to store the received data in the appropriate pattern in memory 22 before reception of that data at the tile 4.

[0135] In the case of receive engine 320, the stride pattern represented by the configuration value in register 830 is used to determine the locations in memory 22 where data received from other tiles 4 of the same processing unit 2 is stored (i.e., during an internal exchange). In the case of transmit engine 300, the stride pattern represented by the configuration value in register 830 is used to determine the locations in memory 22 from which data items are transmitted to tiles 4 of the same processing unit 2 (i.e., during an internal exchange) and tiles 4 of different processing units 2 (i.e., during an external exchange).

[0136] In some cases, it may be desirable to rearrange data held in the memory of a processing device to place the data in a form that is more suitable for subsequent processing. For example, given a set of data held in memory and representing a matrix, it may be desirable to first perform a transpose of the matrix before using the matrix in one or more calculations.

[0137] According to an embodiment, the send and receive mechanisms described herein can be used to perform a rearrangement of data held in memory by activating a loopback mode of processing unit 40. When loopback mode is activated, rather than sending data to an additional processing unit, data can be sent over a loopback path, which returns the data to memory 22 from which it originated. The data can be rearranged based on at least one of i) the order in which the data is sent from memory 22, or ii) the arrangement in which the returned data is stored in memory 22.

[0138] Referring to FIG. 15, there is shown a schematic diagram of various components of a processing unit 40 that may be used to perform data rearrangements within memory 22 by utilizing the send and receive mechanisms described above.

[0139] Processing unit 40 includes control register 120, which stores an indication (which may be referred to as a LOOPBACK field) as to whether data transmitted by transmit engine 300 is transmitted to an external destination on output interface 310 or is transmitted along a bus forming a loopback path through which the data is returned to memory 22 for storage in memory 22. Execution unit 18 is operable to execute instructions to update the value of the LOOPBACK field. Execution unit 18 may update the value of the LOOPBACK field to reflect that loopback mode is enabled, such that any data transmitted after loopback mode is enabled is transmitted over the loopback path. Similarly, execution unit 18 may update the value of the LOOPBACK field to reflect that loopback mode is disabled, such that any data transmitted after loopback mode is disabled is transmitted over interface 310 to external processing unit 40.

[0140] If the LOOPBACK field is set to indicate that data is to be transmitted over the loopback path (i.e., loopback mode is enabled), circuitry 123 of processing unit 40 controls multiplexer 121 in output interface 310 to direct data transmitted from memory 22 to input interface 330 of processing unit 40. Circuitry 123 of processing unit 40 can also control multiplexer 122 in input interface 330 to select the loopback path as an input rather than an input for receiving data from a source external to processing unit 40.

[0141] When loopback mode is enabled, a data unit dispatched from Transmit Engine 300 is transmitted along the loopback path for reception by Receive Engine 320, which stores the data unit in Memory 22 at a location specified by the current value of a write pointer maintained within Receive Engine 320.

[0142] To perform the rearrangement of the set of data units held in memory 22, transmit engine 300 may cause the data units to be transmitted from memory 22 over the loopback path in an out-of-order sequence relative to the order in which the data units are held in memory 22. In other words, the chronological order in which transmit engine 300 transmits the data units over the loopback path does not match the logical arrangement of the data units in memory 22. It is understood that the logical arrangement of the data units in memory 22 is the arrangement of the data units according to logical memory addresses.

[0143] Instead of or in addition to out-of-order transmission by transmit engine 300, receive engine 320 stores the data units in memory 22 out of order relative to the order in which transmit engine 300 transmits the data units over the loopback path. In other words, receive engine 320 stores the data units in memory 22 in a logical arrangement that does not match the chronological order in which receive engine 320 receives the data units.

[0144] As mentioned above, transmit engine 300 is operable to transmit data units according to a variable stride pattern set by execution unit 18. Such a variable stride pattern may be used by transmit engine 300 to cause data units to be transmitted out of order relative to the initial placement of the data units in memory 22. This has the advantage that fewer instructions may need to be executed by execution unit 18 to cause rearrangement of the data units in memory 22.

[0145] As also described above, receive engine 320 is operable to store data units in memory 22 according to a variable stride pattern established by execution unit 18. Such a variable stride pattern may be used by receive engine 320 to store data units in memory 22 in an out-of-place arrangement relative to the order in which the data is received from memory 22 via the internal loopback path.

[0146] Referring to FIG. 16, an example of an operation that may require rearrangement of data within memory 22 is shown, which may be implemented using a scheme in which data is transmitted via a loopback path to perform the rearrangement. The operation is a transpose operation that generates an output matrix 1610 from an input matrix 1600. Assume that the elements of matrix 1600 are stored in memory 22 so that they are arranged consecutively in the order [1, 2, 3, 4, 5, 6, 7, 8, 9] in logical address space. To represent output matrix 1610, the elements in memory 22 must be rearranged to the order [1, 4, 7, 2, 5, 8, 3, 6, 9]. FIGS. 17 and 18 present different examples of how the order in which different locations in memory 22 are accessed when performing a transmit operation or storing received data in memory 22 can be used to perform that data rearrangement. Figure 17 shows an example in which the order in which locations in memory 22 are accessed to transmit data from memory 22 is used to perform the rearrangement of data required for the matrix transpose operation shown in Figure 16. Figure 18 shows an example in which the order in which locations in memory 22 are accessed to store data received via the loopback path is used to perform the rearrangement of data required for the matrix transpose operation shown in Figure 16.

[0147] 17, there is shown a first portion 1700 of memory 22 in which a number of elements are arranged consecutively in a given order, each of the elements (labeled 1-9) representing a single unit of data transmitted within a single data packet within a single processor clock cycle.

[0148] The transmit engine 300 is configured to transmit a first unit (labeled 1) from the memory 22. This first unit is the first in the logical arrangement of data units in the memory portion 1700. The transmit engine 300 causes this first unit to be transmitted from the memory 22 via the loopback path. The receive engine 320 stores this data unit received via the loopback path in a first location in the portion 1710 of the memory 22. The first location is the first memory unit for storing the data unit in the logical arrangement of memory units in the memory portion 1710.

[0149] Following transmission of the first data unit from memory 22, transmit engine 300 causes the next data item to be transmitted from memory 22. This item is non-contiguous with the first data item and, in the example of FIG. 17, is the fourth data item in the logical sequence of data items in portion 1700 of memory 22. Transmit engine 300 causes this data item to be transmitted via the loopback path. Receive engine 320 stores this data item in a second logical location in memory portion 1710.

[0150] Following transmission of the second data item, transmit engine 300 causes a third data item to be transmitted, which is also non-contiguous with the previous item and is seventh in the logical order of items in memory portion 1700. Receive engine 320 causes this data item to be stored in a third location in portion 1710 of memory 22.

[0151] Transmit engine 300 continues to transmit data from memory portion 1700 in the order shown. The fourth data item to be transmitted is second in logical order within portion 1700 of memory 22, and the fifth data item to be transmitted is fifth in logical order within portion 1700 of memory 22. For simplicity, FIG. 17 illustrates only the first five accesses to portion 1700 of memory 22. However, transmit engine 300 continues to access data items in memory 22 according to the order [1, 4, 7, 2, 5, 8, 3, 6, 9]. Because receive engine 320 in this example is configured to store data in memory 22 in the order in which that data is received from transmit engine 300, the resulting logical arrangement within memory portion 1710 matches the order in which transmit engine 300 transmits data.

[0152] 3-11, the order in which data items are transmitted by transmit engine 300 is determined at least in part by the variable stride pattern implemented by transmit engine 300. Specifically, when implementing such a variable stride pattern, transmit engine 300 accesses each of the data items held in memory portion 1700 at a location identified by its read pointer and updates its read pointer between each transmission of the data item. For example, the access pattern applied to portion 1700 of memory 22 used to transmit data items in the described order can be achieved by using the output configuration values ​​shown in Table 2 below.

[0153] [Table 2]

[0154] By repeatedly executing two strides given by +3 and a single stride given by −5, the transmit engine 300 accesses locations in the memory portion 1710 in the order in which the data items are arranged in the memory portion 1710. The transmit engine 300 transmits these items. Given the current selection of the loopback path, these data items are sent to the memory 22 via the loopback path. The receive engine 320 stores these data items along the loopback path in the same order as they were sent by the transmit engine 300. This can be implemented by the receive engine 320 also applying a variable stride pattern, whose input configuration values ​​are such that the inner repeat value is set to a non-zero value (e.g., 1) while the other configuration values ​​are set to zero. The receive engine 320 stores each item received via the loopback path in the location in the memory portion 1710 identified by its write pointer and updates its write pointer by +1.

[0155] In some embodiments, rather than relying entirely on the variable stride function to determine the addresses of all of the source data items, each of the portions of items transmitted from memory portion 1700 may be transmitted in response to execution of a separate SEND instruction (e.g., SEND instruction 900) by execution unit 18, with each such SEND instruction specifying the address of the first data item in memory 22 that is transmitted in response to its execution.

[0156] 18 , there is shown an example in which receive engine 320 is used to implement a rearrangement of data items held within memory portion 1700. In this case, transmit engine 300 transmits each of the data items from memory portion 1700 in an order that matches the logical arrangement in which the data items were stored within memory portion 1700. However, receive engine 320 stores the data items in memory portion 1710 in a new arrangement that does not match the previous logical arrangement of the data items within memory portion 1700.

[0157] Transmit Engine 300 causes a first item (labeled 1) in a first location in Memory Portion 1700 to be transmitted via the loopback path to Receive Engine 320. Upon receiving the first item, Receive Engine 320 stores the item in a first location in Memory Portion 1710.

[0158] Transmit Engine 300 causes a second item (labeled 2) in a second location in memory portion 1700 to be transmitted via the loopback path to Receive Engine 320. Upon receiving this second item, Receive Engine 320 stores the item in a fourth location in memory portion 1710.

[0159] Transmit Engine 300 continues transmitting the items according to their sequential placement in Memory Portion 1700. Receive Engine 320 stores the items in Memory Portion 1710 by applying the access pattern shown in Figure 18. This access pattern mirrors the access pattern applied by Transmit Engine 300 in the example shown in Figure 17.

[0160] The arrangement in which data items are stored by receive engine 320 within memory portion 1710 is determined by the variable stride pattern implemented by receive engine 320 as described above with respect to Figures 3-10. Specifically, receive engine 320 stores each of the data items within memory portion 1710 at a location identified by its write pointer and updates its write pointer between each storage of the data item. For example, the access pattern applied to portion 1710 of memory 22 used to transmit data items in the order described can be achieved by using the input configuration values ​​shown in Table 3 below.

[0161] [Table 3]

[0162] By repeatedly executing two strides of +3 and a single stride given by −5, receive engine 320 accesses locations in memory portion 1710 to store data items in the order shown in Figure 18. This arrangement of memory items represents the transposed form 1610 of matrix 1600.

[0163] In this example, transmission by transmit engine 300 may be performed by applying a variable stride pattern that sets the inner repeat value to a non-zero value (e.g., 1) while setting the other configuration values ​​to zero. Transmit engine 300 causes each of the items in its memory portion 1700 to be transmitted via the loopback path from a location in memory portion 1700 identified by its read pointer. Transmit engine 300 updates its read pointer between each transmission. Receive engine 320 stores each of the items received via the loopback path in a location in memory portion 1710 identified by its write pointer and updates its write pointer according to the input configuration values ​​shown in Table 3.

[0164] In some embodiments, rather than relying entirely on the variable stride function to determine addresses for storing all of the data items, each of the portions of the items stored in memory portion 1710 may be stored in a location determined by execution of a separate memory pointer update instruction (e.g., memory pointer update instruction 1100) by execution unit 18, with each such memory pointer update instruction specifying the address of a location in memory 22 at which the next data item received via the loopback path should be stored.

[0165] An example of one type of operation (i.e., matrix transpose) that requires rearrangement of data has been described with respect to Figures 16-18. However, the same principles are applicable to other types of transformations that may be applied. Furthermore, these examples have described cases in which the rearrangement is performed using either out-of-order transmission (as described with respect to Figure 17) or out-of-order storage (as described with respect to Figure 18). However, in some cases, some rearrangement may be applied by performing both out-of-order transmission and out-of-order storage.

[0166] The transmit functions used to access data and transmit that data via a loopback path or to an external processing unit have been described as being implemented within transmit engine 300. Similarly, the receive functions used to access memory locations and store received data in those memory locations have been described as being implemented within receive engine 320. However, the transmit and receive functions need not be implemented in physically separate blocks. The operations implemented by transmit engine 300 and receive engine 320 can be described as being implemented at a general level by circuitry of processing unit 40.

[0167] The above description describes three different forms of SEND instructions that can be executed by an execution unit: SEND instructions 900, 900a, and 1000. Unless otherwise specified, the teachings herein regarding a "SEND instruction" can be understood to apply to any of these three instructions, as well as to other types of SEND instructions.

[0168] Time-deterministic communication between tiles 4 is described in more detail in prior US Patent Application Publication No. 15 / 886,131, which is incorporated by reference.

[0169] Referring to FIG. 13, a method 1300 for transmitting data according to an embodiment of the present application is shown.

[0170] At S1310, one or more configuration values ​​defining the variable stride pattern are set within the transmit engine, which may be initial values ​​set by execution of one or more put instructions by at least one execution unit before the transmit is performed.

[0171] At S1320, in response to execution of the SEND instruction, the send engine causes the data item at the location in memory identified by the pointer to be sent to at least one destination external to the processing unit.

[0172] In S1330, after transmitting the data item, the transmit engine updates the pointers according to the variable stride pattern.

[0173] At S1340, if the execution unit has reached a point in the compiled code where the variable stride pattern should be updated, then the method 1300 proceeds to S1350. Otherwise, the method 1300 proceeds to S1320 where a further data item is sent.

[0174] At S1350, instructions are executed to update the variable stride pattern. Method 1300 then proceeds to S1320, where additional data items are sent, and S1330, where the pointers are updated according to the new variable stride pattern.

[0175] Referring to FIG. 14, a method 1400 for receiving data according to an embodiment of the present application is shown.

[0176] At S1410, one or more configuration values ​​defining a variable stride pattern are set within the receiving engine, which may be initial values ​​set by execution of one or more put instructions by at least one execution unit before the data item is received.

[0177] At S1420, in response to receiving the data item at a processing unit (eg, tile 4), the receiving engine stores the data item at a location in memory identified by the pointer.

[0178] In S1430, after storing the data item, the receive engine updates the pointer according to a variable stride pattern.

[0179] At S1440, if the execution unit reaches a point in its compiled code where the variable stride pattern should be updated, method 1400 proceeds to S1450.

[0180] At S1450, instructions are executed to update the variable stride pattern, after which the method proceeds to S1420 where further data items are received and stored in memory according to the updated variable stride pattern.

[0181] 19, an example of a method 1900 for rearranging data is shown. Method 1900 includes sending data to an additional processing unit over output interface 310. Prior to the first step in method 1900, control register 120 stores an indication that loopback mode is disabled.

[0182] At S1910, execution unit 18 executes one or more send instructions to cause at least some of the data units held in memory 22 to be sent on output interface 310 to one or more additional processing devices.

[0183] In S1920, execution unit 18 executes an instruction to enable loopback mode, which includes writing an indication that loopback mode is enabled to control register 120.

[0184] Method 1900 then includes steps performed to cause a reordering of a first set of data units held in memory 22. This first set of data units may be data units (labeled 1-9) held in memory portion 1700 shown in FIGS. 17 and 18. Method 1900 shows three separate branches, each representing a different way in which the reordering of data items may be performed. The first branch, represented by the combination of S1930 and S1940, shows one way in which the reordering may be achieved based on out-of-order transmission. The first branch may be applied to perform the reordering shown in FIG. 17. The second branch, represented by the combination of S1950 and S1960, shows another way in which the reordering may be achieved based on out-of-order storage. The second branch may be applied to perform the reordering shown in FIG. 18. The third branch, represented by the combination of S1930 and S1960, shows another way in which the reordering may be achieved based on both out-of-order transmission and out-of-order storage. Each of the three branches involves transmitting the first set of data units from memory 22 along a loopback path so that the first set of data units is stored back within memory 22 .

[0185] At S1930, circuitry of processing unit 40 causes the first set of data units to be transmitted on the loopback path in an out-of-order order relative to the first arrangement in which the first set of data units is maintained in memory 22. This step is performed in response to execution unit 18 executing one or more transmit instructions. Circuitry that implements S1930 includes circuitry of execution unit 18 and transmit engine 300.

[0186] At S1940, circuitry of processing unit 40 stores the first set of data units transmitted on the loopback path at S1930 in memory 22. The circuitry stores each of the first set of data units in memory 22 in the order in which they were transmitted at S1930. Circuitry that implements S1940 includes circuitry of receive engine 320.

[0187] At S1950, circuitry of processing unit 40 causes the first set of data units to be transmitted on the loopback path in an order that corresponds to the first arrangement of the first set of data units maintained in memory 22. This step is performed in response to execution unit 18 executing one or more transmit instructions. Circuitry that implements S1930 includes circuitry of execution unit 18 and transmit engine 300.

[0188] At S1960, circuitry of processing unit 40 stores the first set of data units transmitted on the loopback path at S1950 in memory 22. The circuitry stores each of the first set of data units in memory 22 out of order relative to the order in which the data units were transmitted at S1950. In an implementation, the circuitry that implements S1960 includes circuitry of receive engine 320 and, optionally, execution unit 18.

[0189] It will be understood that the embodiments are described by way of example only.

Claims

1. A processing device, Memory and an execution unit configured to execute a send instruction for sending data external to the processing device; A transmitting engine; Equipped with The transmission engine is one or more registers for storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which the memory is accessed to transmit the data external to the processing device; in response to execution of one or more instances of the send instruction by the execution unit; causing the items of data at locations in the memory identified by the pointers to be transmitted to at least one destination external to the processing device; updating the pointer according to the variable stride pattern between each transmission of one of the plurality of data items; and a processing circuit configured to perform Including, the execution unit is configured to execute at least one instruction to update the variable stride pattern used by the transmit engine; updating the variable stride pattern includes updating at least one of one or more configuration values ​​to one or more new values ​​for use by the transmit engine in performing one or more subsequent transmits in response to further instances of the transmit instruction being executed by the execution unit. Processing equipment.

2. the at least one instruction includes an instruction having a combined sending and updating function; The instruction, when executed by the execution unit, providing the new value of at least one of one or more configuration values ​​to the transmit engine for use by the transmit engine in making one or more subsequent transmissions; transmitting a further data item from said memory; The processing device according to claim 1 , wherein the processing device executes both of the above.

3. 3. The processing device of claim 1, wherein the at least one instruction, when executed by the execution unit, updates the pointer with a previous configuration value and copies a new configuration value into a portion of the one or more registers.

4. 4. The processing device of claim 3, wherein the execution unit is configured to, following the copying of the new values ​​into the portions of the one or more registers, cause the new values ​​to replace the previous values ​​for use by the transmit engine.

5. At least one of the one or more configuration values ​​is: a stride size value defining the separation in said memory between two of said data items transmitted consecutively by said transmit engine; an iteration value defining the number of times a particular size stride is applied by the transmit engine before applying a different size stride; The processing device according to any one of claims 1 to 4, comprising at least one of:

6. the memory is configured to store a compiled set of code including one or more instances of the transmit instructions and one or more instances of at least one instruction for updating the variable stride pattern; the compiled set of code is configured to execute each of the one or more instances of the send instruction during an allotted clock cycle, which is a predetermined number of processor clock cycles after receipt of a synchronization signal; 6. The processing device of claim 1, wherein the compiled set of codes is configured to execute each of one or more instances of at least one instruction for updating the variable stride pattern during an allotted clock cycle that is a predetermined number of processor clock cycles after receipt of the synchronization signal.

7. wiring configured to carry further data from said memory along a loopback path for storage back in said memory; 7. The processing device of claim 1, wherein the processing circuitry of the transmit engine is configured to, in response to execution of one or more additional instances of the transmit instruction by the execution unit, cause the items of the further data to be rearranged in the memory by transmitting the items of the further data over the loopback path in an order determined by a current variable stride pattern active for the transmit engine.

8. A processing device, Memory and An execution unit; A receiving engine; Equipped with The receiving engine one or more registers for storing one or more configuration values ​​defining a variable stride pattern for updating pointers to addresses at which the memory is accessed to store data received by the processing unit; a processing circuit responsive to receiving a plurality of data items at the processing device, storing the plurality of data items in the memory at locations identified by the pointers; updating the pointer according to the variable stride pattern for each reception and storage of one of the plurality of data items; Including, the execution unit is configured to execute at least one instruction for updating the variable stride pattern used by the receive engine; updating the variable stride pattern includes updating at least one of one or more configuration values ​​to one or more new values ​​used by the receiving engine to store one or more further data items received at the processing unit. Processing equipment.

9. the at least one instruction includes an instruction having a combined sending and updating function; The instruction, when executed by the execution unit, providing at least one new value of one or more of the configuration values ​​to the receiving engine for use by the receiving engine in storing one or more further data items; transmitting an output data item from said memory; The processing device according to claim 8 , wherein the processing device executes both of the above.

10. 10. A processing device as claimed in claim 8 or 9, wherein the at least one instruction, when executed by the execution unit, updates the pointer with a previous configuration value and copies a new configuration value into a portion of one or more registers.

11. 11. The processing device of claim 10, wherein the execution unit is configured to cause the new value to replace a previous value for use by the receiving engine following copying the new value into a portion of one or more registers.

12. At least one of the one or more configuration values ​​is: a stride size value defining the separation in said memory between locations in said memory where data items are stored consecutively by said receiving engine; an iteration value defining the number of times a particular size stride is applied by the receive engine before applying a different size stride; The processing device according to any one of claims 8 to 11, comprising at least one of:

13. the memory is configured to store a compiled set of code including one or more instances of at least one instruction for updating the variable stride pattern; 13. The processing device of claim 8, wherein the compiled code set is configured such that each of one or more instances of at least one instruction is executed by the execution unit during an allotted clock cycle that is a predetermined number of processor clock cycles after receipt of a synchronization signal at the processing device.

14. wiring configured to transmit further data from said memory along a loopback path for storage back in said memory; 14. A processing device according to any one of claims 8 to 13, wherein the processing circuitry of the receive engine is configured to, in response to receiving a plurality of items of further data via the loopback path, cause the plurality of items of further data to be rearranged in the memory by storing the plurality of items of further data in an arrangement that depends on a current variable stride pattern active for the receive engine.

15. The processing device of any preceding claim, wherein said at least one of said one or more configuration values ​​comprises a plurality of configuration values.

16. the execution unit is configured to update a string of bits held in a configuration register of the processing unit upon execution of at least one instruction; the string of bits held in the configuration register is divided into a plurality of subsets of bits each of which is mapped to one of the plurality of configuration values; 16. The processing device of claim 15, wherein the processing circuitry is configured, in response to writing the string of bits to the configuration register, to use updated values ​​for those of the plurality of configuration values ​​to which the subset is mapped.

17. 17. The processing device of claim 16, wherein each of the plurality of subsets of bits is shorter in length than a corresponding value of the plurality of configuration values ​​to which it is mapped.

18. The processing device of any one of claims 15 to 17, wherein the plurality of configuration values ​​comprises a plurality of stride size values.

19. 20. The processing device of claim 18, wherein the plurality of configuration values ​​comprises a plurality of repetition values, each of the repetition values ​​being associated with one of the stride size values ​​and specifying the number of times a corresponding value of the stride size value is applied before the processing circuitry applies a different one of the stride size values.

20. A processing device according to any preceding claim, wherein the one or more registers are control and status registers belonging to a thread executing on the execution unit.

21. A multi-tile processing unit comprising a plurality of instances of a processing device according to any preceding claim, wherein each of the plurality of instances of the processing device is a tile.

22. 1. A method for transmitting data from a processing device, comprising: a transmit engine of the processing device, storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which memory is accessed to transmit data external to the processing device; In response to execution of one or more instances of the transmit instruction, causing a plurality of data items at locations in said memory identified by said pointers to be transmitted to at least one destination external to said processing device; updating the pointer according to the variable stride pattern between each transmission of one of the plurality of data items; and Including, an execution unit of the processing unit executing at least one instruction for updating the variable stride pattern; updating the variable stride pattern includes updating at least one of the one or more configuration values ​​to one or more new values ​​for use in making one or more subsequent transmissions in response to execution of a further instance of the transmit instruction.

23. 1. A method of storing data received by a processing device, comprising: a receiving engine of the processing device, storing one or more configuration values ​​defining a variable stride pattern for updating pointers to addresses at which memory is accessed to store data received at said processing unit; In response to receiving a plurality of data items at said processing device, storing the plurality of data items in the memory at locations identified by the pointers; updating the pointer according to the variable stride pattern for each reception and storage of one of the plurality of data items; and Including, an execution unit of the processing unit executing at least one instruction for updating the variable stride pattern; updating the variable stride pattern includes updating at least one of the one or more configuration values ​​to one or more new values ​​used to store one or more further data items received at the processing device.

24. 1. A computer program comprising a set of computer readable instructions which, when executed by an execution unit of a processing device, causes the computer to perform a method for transmitting data from said processing device, said method comprising: storing one or more configuration values ​​defining a variable stride pattern for updating a pointer to an address at which memory is accessed to transmit data external to the processing device; In response to execution of one or more instances of the transmit instruction, causing a plurality of data items at locations in said memory identified by said pointers to be transmitted to at least one destination external to said processing device; updating the pointer according to the variable stride pattern between each transmission of one of the plurality of data items; and Including, The method further includes updating the variable stride pattern in response to execution of at least one instruction to update the variable stride pattern; updating the variable stride pattern includes updating at least one of the one or more configuration values ​​to one or more new values ​​for use in making one or more subsequent transmissions in response to execution of further instances of the transmit instructions.

25. 1. A computer program comprising a set of computer readable instructions which, when executed by an execution unit of a processing device, causes said computer to perform a method of storing data received at said processing device, said method comprising: storing one or more configuration values ​​defining a variable stride pattern for updating pointers to addresses at which memory is accessed to store data received at said processing unit; In response to receiving a plurality of data items at said processing device, storing the plurality of data items in the memory at locations identified by the pointers; updating the pointer according to the variable stride pattern for each receipt and storage of one of the plurality of data items; and Including, The method further includes updating the variable stride pattern in response to execution of at least one instruction to update the variable stride pattern; updating the variable stride pattern includes updating at least one of the one or more configuration values ​​to one or more new values ​​used to store one or more further data items received by the processing device.

Images