Executing concurrent threads on reconfigurable processing grid

Abstract

To provide a system for processing a plurality of concurrent threads.SOLUTION: A system 200 comprises: a reconfigurable processing grid 210 comprising each thread context for one of a plurality of concurrent threads, each implementing a dataflow graph including an identified operation; and a hardware processor 201 configuring the reconfigurable processing grid for: executing a first thread of the plurality of concurrent threads; and while executing the first thread: storing a runtime context value of the first thread in a context storage; while waiting for completion of the identified operation by identified logical elements, executing the identified operation of a second thread by the identified logical element; and when execution of the identified operation of the first thread completes: acquiring the runtime context value of the first thread from the context storage; and executing another operation of the first thread.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] [Related Applications] This application claims the benefit of priority to U.S. Patent Application No. 18 / 218,152, filed July 5, 2023, the contents of which are incorporated herein by reference in their entirety.

[0002] This application is also related to U.S. Patent Application No. 18 / 409,869, filed July 5, 2023, the contents of which are incorporated by reference in their entirety as if fully set forth herein.

[0003] Certain embodiments described in this disclosure relate to a reconfigurable processing grid, and more particularly, but not exclusively, to executing one or more dataflow graphs on a reconfigurable processing grid. [Background technology]

[0004] As used herein, the term reconfigurable processing grid refers to a processing circuit including a plurality of reconfigurable logic elements connected by a plurality of reconfigurable data path control branches, where the plurality of reconfigurable logic elements and additionally or alternatively the plurality of reconfigurable data path control branches may be operated to perform one or more operations in each of one or more iterations. As used herein, the term data flow refers to a computer programming paradigm that models at least a portion of a software program as a directed graph (data flow graph) of data flowing between operations such that a sequence of operations is applied to each data element in a sequence of data elements of the data flow graph. Optionally, the data flow graph comprises a plurality of nodes, each of which applies an operation to a data element, and a plurality of directed edges, each of which connects two of the plurality of nodes and indicates a flow of data between the two nodes. In the field of computer science, an execution thread is a sequence of computer instructions that can be independently managed by a scheduler. For simplicity, the term "thread" is used in the sense of "thread of execution", and the terms are used interchangeably herein. A thread may implement a data flow graph. As used herein, the term "projection" refers to the process of manipulating one or more reconfigurable logic elements of a reconfigurable processing grid, and additionally or alternatively, manipulating one or more reconfigurable datapath branches of the reconfigurable processing grid, to execute a dataflow graph. Thus, projecting a thread implementing a dataflow graph onto a reconfigurable processing grid refers to configuring the reconfigurable processing grid by manipulating one or more reconfigurable logic elements of the reconfigurable processing grid, and additionally or alternatively, manipulating one or more reconfigurable datapath branches of the reconfigurable processing grid, to execute the dataflow graph implemented by the thread.

[0005] In computer science, concurrent computing refers to the simultaneous execution of multiple threads of execution of a software program. Executing multiple threads of a software program simultaneously can improve the overall performance and responsiveness of a system. Metrics used to measure the performance of a system include, but are not limited to, the amount of tasks performed by the system in a specified amount of time (throughput), the amount of time it takes for the execution of a task to be completed (latency), and the amount of computer memory the system uses when operating. Concurrent computing can be used to increase the throughput and reduce the latency of a system.

[0006] Each of the multiple concurrent threads may comprise one or more specified operations. When multiple concurrent threads are executed simultaneously on a reconfigurable processing grid, each of the multiple concurrent threads is projected onto a portion of the reconfigurable processing grid, i.e., a portion of the multiple logic elements of the reconfigurable processing grid is operated to execute the concurrent thread, e.g., to execute a dataflow graph implemented by the concurrent thread.

[0007] There are computer instructions that have inconsistent latencies to complete. Such operations may require different amounts of time to execute when executed multiple times, and may additionally or alternatively require more time to complete than other instructions. Examples of such operations with inconsistent latencies include, but are not limited to, memory accesses, accesses to peripheral devices, execution of computational kernels, etc. Summary of the Invention [Means for solving the problem]

[0008] It is an objective of some embodiments described in this disclosure to provide a system and method for executing multiple concurrently executing threads by storing multiple thread contexts, each for one of the multiple concurrently executing threads, in a context storage and using the context storage to manage the execution of the multiple concurrently executing threads on a reconfigurable processing grid. Optionally, one or more logic elements operated to execute an identified operation of a first thread are used without reconfiguring one or more logic elements to execute an identified operation of a second thread while the first thread is pending completion of its identified operation. Optionally, the identified operations have inconsistent latency.

[0009] These and other objects are achieved by means of the features of the independent claims. Further embodiments are evident from the dependent claims, the description and the drawings.

[0010] According to a first aspect, a system for processing a plurality of concurrently executing threads includes at least one reconfigurable processing grid comprising a plurality of logic elements and a context storage configured to store a plurality of thread contexts, each thread context being for one of a plurality of concurrently executing threads, each concurrently executing thread implementing a dataflow graph comprising a plurality of operations including at least one identified operation, each of the plurality of thread contexts comprising, for that concurrently executing thread, at least one runtime context value of a dataflow graph implemented by the concurrently executing thread; and at least one hardware processor for configuring said at least one reconfigurable processing grid, The system is configured to execute a first thread of the row threads, and while executing the first thread, store at least one runtime context value of the first thread in a context storage, execute at least one identified operation of a second thread of the plurality of threads by the at least one identified logic element while waiting for completion of execution of at least one identified operation of the plurality of operations of the first thread by the at least one identified logic element of the plurality of logic elements, and retrieve at least one runtime context value of the first thread from the context storage when execution of the at least one identified operation of the first thread is completed, and execute at least one other operation of the plurality of operations of the first thread. Storing thread contexts in the context storage for each of the plurality of concurrently executing threads allows for pausing and resuming execution of one or more of the plurality of concurrently executing threads without manipulating the processing grid, thus reducing the complexity of reducing the amount of time that the one or more identified logic elements are idle waiting for completion of execution of one or more operations of the first thread. This increases the throughput of the system and reduces the latency of the system in executing its one or more tasks.

[0011] According to a second aspect, a method for processing a plurality of concurrently executing threads includes executing a first thread of the plurality of concurrently executing threads, each concurrently executing thread implementing a data flow graph comprising a plurality of operations comprising at least one identified operation; and configuring a plurality of thread contexts to be stored while executing the first thread. storing at least one runtime context value of a dataflow graph performed by a first thread in an identified context storage, each thread context being a thread context for one of a plurality of concurrently executing threads, each of the plurality of thread contexts including, for that concurrently executing thread, at least one runtime context value of a dataflow graph performed by the first thread; executing at least one identified operation of a second thread of the plurality of threads by the at least one identified logic element while waiting for completion of execution of the at least one identified operation of the plurality of operations of the first thread by the at least one identified logic element of the plurality of logic elements; obtaining the at least one runtime context value of the first thread from the context storage when execution of the at least one identified operation of the first thread is completed; and executing at least one other operation of the plurality of operations of the first thread.

[0012] According to a third aspect, a software program product for executing a plurality of concurrently executing threads includes a non-transitory computer readable storage medium; first program instructions for executing a first thread of the plurality of concurrently executing threads, each concurrently executing thread implementing a dataflow graph comprising a plurality of operations comprising at least one identified operation; and storing, during execution of the first thread, at least one runtime context value of the dataflow graph implemented by the first thread in a context storage, the context storage being configured to store a plurality of thread contexts, each thread context being a thread context for one of the plurality of concurrently executing threads, each of the plurality of thread contexts representing, for that concurrently executing thread, a dataflow graph implemented by the first thread, the dataflow graph comprising a plurality of operations comprising at least one identified operation. and second program instructions including: storing at least one runtime context value of the graph; executing, by the at least one identified logic element, at least one identified operation of a second thread of the plurality of threads while waiting for completion of execution of the at least one identified operation of the plurality of operations of the first thread by the at least one identified logic element of the plurality of logic elements; and upon completion of execution of the at least one identified operation of the first thread, retrieving from the context storage the at least one runtime context value of the first thread; and executing at least one other operation of the plurality of operations of the first thread, wherein the first and second program instructions are executed by the at least one computerized processor from a non-transitory computer-readable storage medium.

[0013] Referring to the first and second aspects, in a first possible implementation of the first and second aspects, the context storage comprises a plurality of context entries, each context entry for storing a plurality of runtime context values ​​of one of the plurality of thread contexts. Using the context entries to store a plurality of runtime context values ​​of one of the plurality of thread contexts allows each thread of the plurality of threads to be restored independently of the other threads, increasing flexibility in the use of the one or more identified logic elements and thus reducing the idle time the one or more identified logic elements spend waiting for completion of execution of one or more operations of the first thread.

[0014] With reference to the first and second aspects, or a first implementation of the first and second aspects, in a second possible implementation of the first and second aspects for at least one thread of the multiple concurrently executing threads, the multiple runtime context values ​​of the at least one thread are stored in multiple context entries of the multiple context entries. By using multiple context entries to store the multiple runtime context values ​​of a thread, a thread can have a large context with an amount of runtime context values ​​that do not fit in a single entry, improving usability of the system compared to storing multiple runtime context values ​​of a thread in a single entry. Optionally, the multiple context entries are organized in a table having multiple rows, one for each of the multiple context entries. Optionally, each row of the multiple rows has multiple columns, and each of the multiple runtime context values ​​of the thread context stored in a row is stored in one of the multiple columns. Organizing the multiple context entries in a table improves usability of the multiple context entries, allowing reference to values ​​by an index number of the entry, and additionally or alternatively, by an index number of the column. Optionally, the at least one reconfigurable processing grid is further configured to store at least one runtime context value of the first thread in at least one identified column of the context storage while executing the first thread, and to store at least one other runtime context value of the first thread in at least one identified column of the context storage. By reusing context entries, an amount of storage required to implement the context storage may be reduced, which may reduce implementation costs. Optionally, the dataflow graph comprises a plurality of nodes and a plurality of edges. Optionally, at least one node of the plurality of nodes implements a lookup table, and configuring the at least one reconfigurable processing grid to execute the first thread comprises storing the lookup table in at least one other column of the plurality of columns.Storing the lookup table in one or more columns of the context storage provides faster access to lookup table values ​​than implementation in application memory, reducing the time it takes a thread to access lookup table values, and additionally or alternatively reducing the time it takes to create a context for a thread when a context value is driven from the lookup table.

[0015] With reference to the first and second aspects or a first implementation example of the first and second aspects, in a third possible implementation example of the first and second aspects, the at least one reconfigurable processing grid is further configured to, upon completion of execution of the at least one identified operation of the first thread, store at least one result value resulting from execution of the at least one identified operation of the first thread in a context entry of the plurality of context entries, the context entry being intended for storing at least a portion of a thread context of the first thread. Storing the result value in the context entry increases the accuracy of the context of the thread when resumed after completion of execution of the one or more identified operations.

[0016] With reference to the first and second aspects or the first implementation of the first and second aspects, in a fourth possible implementation of the first and second aspects, a first context entry of the multiple context entries stores multiple runtime context values ​​of a first thread, and the at least one reconfigurable processing grid is configured to calculate specific information that the first context is completed according to a result of the at least one test applied to the multiple runtime context values ​​of the first context entry, and to obtain the at least one runtime context value of the first thread and to perform at least one other operation conditional on the specific information that the first context is completed. Optionally, the at least one reconfigurable processing grid further comprises a dispatch circuit for applying the at least one test to the multiple runtime context values. Optionally, applying the at least one test to the multiple runtime context values ​​comprises the dispatch circuit executing a set of test instructions. Using a dispatch circuit that executes a set of test instructions allows multiple tests to be performed by executing a different set of test instructions for each test, facilitating increasing the accuracy of the results of applying one or more tests and thus increasing the accuracy of the identification information that the first context is completed. Optionally, the first context entry comprises a plurality of validity bits, each associated with one of the plurality of runtime context values, and applying at least one test to the plurality of runtime context values ​​comprises applying a specified bitwise mask to the plurality of validity bits. Applying the bitwise mask to the plurality of validity bits reduces the time required to check the plurality of validity values ​​of the plurality of runtime context values. Optionally, the at least one reconfigurable processing grid is further configured to select a first thread for executing at least one other operation of the plurality of operations in accordance with a dispatch policy.Selecting the first thread by the one or more reconfigurable processing grids reduces latency to execution of the one or more other operations compared to selecting the first thread by processing circuitry external to the processing grid, e.g., when the one or more hardware processors are not part of the one or more processing grids. Optionally, the at least one reconfigurable processing grid is further configured to calculate other identifying information that the at least one other context is completed according to at least one other result of applying at least one test to the at least one other runtime context value of the at least one other context entry before selecting the first thread. Identifying that the one or more other contexts are completed before selecting the first thread allows flexibility in selecting a thread to execute, e.g., the other thread whose context is the one or more other contexts, facilitating improved overall system performance compared to being limited to selecting only the first thread. Optionally, the at least one reconfigurable processing grid is further configured to at least one of: deny execution of at least one other operation of the plurality of operations of the first thread, and provide at least one of the plurality of thread context values ​​of the first thread to at least one other software object, conditional on a mark added to one or more of the plurality of context entries, the context entries being for storing at least a portion of a thread context of the first thread. The use of the mark allows flexibility in selecting who executes which thread, facilitating improved overall system performance compared to being limited to resuming execution of the first thread and terminating the thread after resumption.

[0017] With reference to the first and second aspects, in a fifth possible implementation of the first and second aspects, the dataflow graph comprises a plurality of nodes and a plurality of edges. Optionally, the at least one identified operation is represented in the dataflow graph by at least one identified node of the plurality of nodes. Optionally, the at least one hardware processor is further configured to identify a subgraph (a residual subgraph) in the dataflow graph, the subgraph being composed of a subset of the nodes of the plurality of nodes and a subset of the edges of the plurality of edges, the residual subgraph being such that no path exists in the dataflow graph between any two nodes of the at least one identified node, for each node of the subset of nodes, no path exists in the dataflow graph between the at least one identified node and the node, for each edge of the subset of edges, no path exists in the dataflow graph between the at least one identified node and the edge, and the at least one runtime context value is at least one edge value of the subset of edges. Using a residual subgraph as the context for the flow, where there is no path in the dataflow graph between one or more identified nodes and any node in the residual subgraph, increases the accuracy of the context because when execution of one or more identified operations is completed, execution of other parts of the dataflow graph that are not part of the residual subgraph does not affect execution of the residual subgraph, and vice versa. Optionally, the dataflow graph is a directed graph. Optionally, each of the plurality of edges has a head node of the plurality of nodes and a tail node of the plurality of nodes, the subset of nodes comprises one or more entry nodes, such that each of the one or more entry nodes is an entry node of the residual subgraph where the entry node is not a head node of any of the subset of edges, and the at least one runtime context value is at least one input value of at least one of the one or more entry nodes.

[0018] With reference to the first and second aspects, in a sixth possible implementation example of the first and second aspects, the at least one identified operation comprises at least one of a memory access operation, a floating point mathematical operation, execution of other computational graphs, access to a coprocessor, and access to a peripheral device connected to the at least one reconfigurable processing grid.

[0019] With reference to the first and second aspects, in a seventh possible implementation of the first and second aspects, the multiple concurrently executing threads are a subset of a set of concurrently executing threads, each of the set of concurrently executing threads implementing a data flow graph. Optionally, the system further comprises a buffer storage for storing a plurality of other thread contexts for at least one of the set of concurrently executing threads, respectively. Optionally, the at least one hardware processor is configured to: store in the buffer storage one or more additional runtime context values ​​of one or more waiting threads, the one or more waiting threads being not members of the multiple concurrently executing threads, to further configure the at least one reconfigurable processing grid; determine in each of the multiple iterations that execution of the at least one additional thread of the multiple concurrently executing threads has been completed; retrieve from the buffer storage the at least one additional runtime context value for at least one of the one or more waiting threads; and add the at least one waiting thread to the multiple concurrently executing threads for execution by the multiple logic elements. By using the buffer storage to store one or more additional runtime context values ​​for waiting threads that are not members of the multiple concurrently executing threads, logic elements may be reused to execute more threads than are simultaneously supported by the context storage at one time, further improving overall system performance in terms of reducing latency and additionally or alternatively increasing throughput. Optionally, the at least one reconfigurable processing grid further comprises a register circuit for tracking the one or more waiting threads and an additional dispatch circuit for managing the execution of the multiple concurrently executing threads.Optionally, the additional dispatch circuitry is configured to select at least one waiting thread from the register circuitry, obtain at least one additional runtime context value of the at least one waiting thread from the buffer storage, and add the at least one waiting thread to the multiple concurrently executing threads for execution by the multiple logic elements. Using the register circuitry for tracking the one or more waiting threads and the additional dispatch circuitry for managing execution of the multiple concurrently executing threads reduces latency in scheduling one or more threads of the multiple concurrently executing threads for execution compared to managing execution by one or more hardware processors. Optionally, adding the at least one waiting thread to the multiple concurrently executing threads includes storing the at least one additional runtime context value of the at least one waiting thread in a context storage. Optionally, the additional dispatch circuitry is further configured to associate with each of the at least one waiting thread a context identification value indicative of a thread context of the waiting thread in the context storage. Optionally, the additional dispatch circuitry is further configured to: associate the determined context identification value with a first one of the one or more waiting threads in a first iteration of the multiple iterations; and identify that execution of the first waiting thread has been completed and associate the determined context identification value with a second one of the one or more waiting threads in a second iteration of the multiple iterations. Optionally, the buffer storage comprises a plurality of buffer entries, each storing a thread context of at least one of the one or more waiting threads. Optionally, the register circuitry comprises a plurality of register entries, each for tracking at least one of the one or more waiting threads.Optionally, the additional dispatch circuitry is further configured to: generate, for the at least one group of waiting threads of the one or more waiting threads, a common thread context associated with each of the group of the at least one waiting thread in the buffer storage; generate, in the register circuitry, a common register entry associated with each of the group of the at least one waiting thread; and, when selecting a new thread of the group of the at least one waiting thread from the register circuitry as the at least one waiting thread, calculate at least one private context value of the new thread. Optionally, the register circuitry is further configured to stall execution of at least a portion of the set of concurrently executing threads until an amount of the one or more waiting threads exceeds a threshold. Waiting for an amount of the one or more waiting threads to exceed a threshold allows one or more waiting threads to be added to the multiple concurrently executing threads in batches, reducing the overhead of such configuration and facilitating further improvement of system performance compared to executing waiting threads as they become available.

[0020] With reference to the first and second aspects, in an eighth possible implementation example of the first and second aspects, the plurality of logic elements are a plurality of reconfigurable logic elements organized into a plurality of computational groups, and the at least one identified logic element is a subset of the plurality of computational groups.

[0021] Referring to the first and second aspects, in a ninth possible implementation example of the first and second aspects, the runtime context values ​​of the dataflow graph are input or output values ​​of nodes of the dataflow graph.

[0022] With reference to the first and second aspects, in a tenth possible implementation of the first and second aspects, the at least one hardware processor is further configured to configure at least one reconfigurable processing grid to execute the first thread in each of the multiple thread iterations. Optionally, the context storage comprises at least one additional context entry for storing additional runtime context values ​​of the dataflow graph, the additional runtime context values ​​being common to the multiple thread iterations, and upon completion of execution of the at least one identified operation of the first thread, the reconfigurable processing grid is further configured to retrieve at least one of the additional runtime context values ​​from the context storage. Using one or more additional context entries to store additional context values ​​common to the multiple thread iterations allows for a reduced size of the context storage and a reduced cost of implementation compared to duplicating the additional context values ​​for multiple of the multiple thread iterations.

[0023] With reference to the first and second aspects, in an eleventh possible implementation of the first and second aspects, the at least one reconfigurable processing grid further comprises at least one other context storage. Optionally, the at least one hardware processor is configured to: store at least one additional runtime context value of the first thread in the at least one other context storage when execution of the at least one identified operation of the first thread is completed, further execute the first thread, and while further executing the first thread, execute by the at least one other identified logic element of a second other thread of the plurality of threads while waiting for completion of further execution of the at least one other identified operation of the plurality of operations of the first thread by the at least one other identified logic element of the plurality of logic elements. The use of the one or more context storages allows for cascading separate scheduling for one or more unpredictable latency operations, which can further improve system performance in terms of reduced latency and additionally or alternatively increased throughput.

[0024] With reference to the first and second aspects, in a twelfth possible implementation of the first and second aspects, the at least one reconfigurable processing grid further comprises at least one counter, the plurality of concurrently executing threads comprises a group of concurrently executing threads associated with the at least one counter, and the at least one runtime context value comprises at least one counter value read from the at least one counter by accessing the at least one counter. Maintaining a common counter associated with the group of concurrently executing threads can provide a unique value for each of the group of concurrently executing threads when other context values ​​are common, improving accuracy of operations of the group of concurrently executing threads. Optionally, each of the group of concurrently executing threads implements the identified data flow graph. Optionally, the access to the at least one counter comprises an atomic access comprising reading the at least one counter and incrementing the at least one counter. Optionally, the at least one counter is a sequence of counters. Optionally, incrementing the at least one counter comprises incrementing a first counter of the sequence of counters using modular arithmetic and incrementing a second counter consecutive to the first counter in the sequence of counters, conditional on the first counter wrapping around after being incremented. Optionally, the at least one counter is a sequence of counters, and incrementing the at least one counter comprises at least one of incrementing a first counter of the sequence of counters, incrementing a second counter consecutive to the first counter in the sequence of counters, conditional on the first counter exceeding a maximum value, setting the first counter to a new value calculated using the second counter, and setting the maximum value to another new value calculated using the second counter.

[0025] Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following figures and detailed description. All such additional systems, methods, features and advantages are intended to be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.

[0026] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the embodiments pertain. Although methods and materials similar or equivalent to those described herein may be used in carrying out or testing the embodiments, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, shall prevail. Moreover, the materials, methods, and examples are illustrative only and are not necessarily intended to be limiting. [Brief description of the drawings]

[0027] Certain embodiments are described herein, by way of example only, with reference to the accompanying drawings. Referring specifically and in detail herein to the drawings, it is emphasized that the particulars shown are by way of example and for the purposes of illustrative discussion of the embodiments. In this regard, the description taken together with the drawings will make apparent to those skilled in the art how the embodiments may be practiced.

[0028] [Figure 1] FIG. 1 is a schematic block diagram of an exemplary data flow graph in accordance with some embodiments. [Diagram 2] FIG. 2 is a schematic block diagram of an example system, according to some embodiments. [Diagram 3] FIG. 3 is a schematic block diagram of an exemplary context storage, according to some embodiments. [Figure 4] FIG. 4 is a schematic block diagram of another exemplary system, according to some embodiments. [Diagram 5]FIG. 5 is a flow chart that generally depicts an optional flow of operations according to some embodiments. [Figure 6] FIG. 6 is a flow chart that generally illustrates an optional computational flow for a processing grid, according to some embodiments. [Figure 7] FIG. 7 is a flow chart that generally illustrates another optional computational flow for a processing grid, according to some embodiments. [Figure 8] FIG. 8 is a flow chart that generally depicts yet another optional computational flow for a processing grid, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] In the field of computer programming, the term "context" refers to the state that exists when an operation is executed. In the context of a thread, the context of a thread is the set of values ​​that are accessible when the thread executes an operation. When a thread is executed by a Central Processing Unit (CPU), the context of a thread includes, among other values, values ​​in the CPU's registers, one or more values ​​in Thread Local Storage (TLS), values ​​in stack memory, and the value of the program counter.

[0030] If a first thread involves operations with inconsistent latency, the CPU may remain idle while waiting for the operations to complete. To improve the performance of a system that executes threads, it is a common practice to configure a CPU to execute a second thread while waiting for the operations of the first thread to complete. Methods exist to save the context of a thread when the execution of the thread by the CPU is suspended and to restore the context of a thread when the execution of the thread by the CPU is resumed. When a thread is executed by a CPU, values ​​accessible to the thread reside in the CPU's registers or are accessible via an address bus. Thus, the context of a thread executed by a CPU is determined by the resources available to the CPU and is typically similar for all threads executed by the CPU. Regardless of the thread's function, the context of a thread includes values ​​for a specified set of registers and a specified size of stack memory.

[0031] The growing demand for high performance computerized systems that provide high throughput and low latency has led to an increased use of massively parallel programming paradigms in implementing software applications. In such programming paradigms, a large program is divided into multiple smaller problems that can be solved in parallel, and each problem is solved separately without significant data dependencies and, additionally or alternatively, without dependencies on control between the multiple smaller problems. A software program (program) may contain one or more such parallel regions, and the program's execution is divided from a single thread that executes sequentially (serially) into multiple concurrent threads that each perform the task of solving a smaller problem. This is also known as a fork operation. When the multiple concurrent threads are completed, the program's execution is returned to the single thread that executes serially. This process is known as combining the multiple concurrent threads into a single serial thread.

[0032] While there is no exact threshold amount of concurrent threads that determines when a computation becomes massively parallel, for a given system there is an amount of concurrent threads at which the creation of threads and process contexts and, additionally or alternatively, the execution of the concurrent threads (e.g., when there are more concurrent threads than processing circuitry) requires significant processing resources in terms of memory and computation time, resulting in increased overhead in program execution.

[0033] One existing solution to reduce context switching overhead is the use of a dataflow graph in implementing a distributed method, for example in a distributed implementation that executes a loop. By using multiple concurrent threads, each of which implements a dataflow graph, each of the multiple concurrent threads can be projected onto a portion of a reconfigurable processing grid for concurrent execution. Furthermore, since the multiple concurrent threads implement a dataflow graph common to each thread that executes using the thread's data, one projection of the dataflow graph onto the processing grid can be used to execute multiple threads of the multiple concurrent threads in a pipeline that provides the thread's input data to each thread's dataflow graph. The use of a dataflow graph in the processing grid allows distributed processing to be scaled to a greater extent than is possible using multiple CPUs, at least in part due to the nature of the pipeline executing at least some of the multiple concurrent threads on a projection of the dataflow graph in the processing grid. However, a system may comprise more threads than can be executed simultaneously on the processing grid.

[0034] Furthermore, the pipelined nature of dataflow graph execution is such that when a dataflow graph includes an operation with inconsistent latency (an inconsistent latency operation), execution of other portions of the dataflow graph is stalled while waiting for the inconsistent latency operation to complete for a thread of the multiple concurrently executing threads. Such other portions may include at least some of the threads of the multiple concurrently executing threads executed by the projection of the dataflow graph onto a processing grid. As a result, execution of one or more of the at least some threads may be delayed, reducing the throughput of the system while potentially idling processing resources (within the processing grid).

[0035] Furthermore, when a thread implementing a dataflow graph is projected onto a processing grid, values ​​accessible to the thread may be located on any logical element of the processing grid and are determined by the projection of the thread. When a dataflow graph includes operations with inconsistent latency (inconsistent latency operations), one or more logical elements operated to execute the thread cannot be reused to execute other threads implementing other dataflow graphs without being operated on again. Typically, the amount of time waiting for the inconsistent latency operations of a first thread to complete is sufficient to execute a second thread, but not enough to reconfigure the processing grid, execute the second thread, and reconfigure the processing grid again to resume execution of the first thread. As a result, processing resources (within the processing grid) may be idle while execution of the second thread is delayed, reducing the throughput of the system.

[0036] The present disclosure, in at least some embodiments thereof, addresses the technical problem of reducing the amount of time that processing resources in a processing grid are idle, e.g., while waiting for operations of inconsistent latency to complete execution, particularly when executing multiple concurrent threads in a pipeline that implement a dataflow graph of one or more identified operations. By mitigating the amount of time that processing resources in a processing grid are idle, at least some embodiments described herein improve system performance, e.g., reduce system latency, and additionally or alternatively improve system throughput, as compared to standard approaches for executing multiple concurrent threads on a processing grid.

[0037] To that end, in some embodiments thereof, the present disclosure proposes storing thread context for each of a plurality of concurrently executing threads in a context storage, and using the context storage to suspend and resume execution of one or more of the plurality of concurrently executing threads without manipulating the processing grid.

[0038] Unless otherwise stated, hereafter, for brevity, the term "context" means "thread context" and the terms are used interchangeably. Optionally, the thread context comprises a set of values, each of which is an input value to a node of the dataflow graph or an output value of a node of the dataflow graph. In such an embodiment, the present disclosure proposes storing in a context storage one or more runtime context values ​​of a first thread executing a dataflow graph comprising one or more identified operations. A runtime context value of a thread is a value accessible to a thread while the thread is executing. Optionally, at least one of the one or more identified operations is an operation of inconsistent latency. Optionally, while waiting for completion of execution of the one or more identified operations of the first thread by one or more identified logic elements of a plurality of logic elements of the processing grid, the present disclosure proposes execution of one or more identified operations of a second thread by the one or more identified logic elements. Optionally, the second thread is executed by the one or more identified logic elements concurrently with the first thread in the pipeline. Thus, instead of one or more identified logic elements remaining idle while waiting for completion of one or more identified operations of a first thread, and additionally or alternatively, instead of one or more logic elements simultaneously remaining idle because the entire pipeline is stalled, one or more logic elements can be used to execute one or more identified operations of a second thread.

[0039] There is no need to reconfigure the processing grid, i.e. to manipulate the one or more identified logic elements, since they are already configured to execute the one or more identified operations. Optionally, upon completion of the execution of the one or more identified operations of the first thread, the present disclosure proposes retrieving one or more runtime context values ​​of the first thread from the context storage and resuming the execution of the first thread, i.e., executing one or more other operations of the first thread. Optionally, the one or more runtime context values ​​are loaded into one or more logic elements of the processing grid before executing the one or more identified operations of the second thread. By storing one or more runtime context values ​​of the first thread in the context storage before executing the one or more identified operations of the second thread and retrieving the one or more runtime context values ​​when the execution of the one or more identified operations of the first thread is completed, the context of the first thread can be preserved such that resuming its execution after the execution of the one or more identified operations is not affected by executing the one or more identified operations of the second thread using the one or more identified logic elements. This provides the advantage of reducing the amount of time that the one or more identified logic elements are idle waiting for execution of one or more operations of the first thread to complete, facilitating increased throughput of the system and reduced latency of the system in executing the one or more tasks.

[0040] Optionally, the one or more identified operations are represented in the dataflow graph by one or more identified nodes of the plurality of nodes of the dataflow graph. The context of the one or more identified operations comprises a subgraph of the dataflow graph (the residual subgraph) that does not include paths leading to or from the one or more identified nodes. Thus, the residual subgraph comprises a subset of the plurality of nodes of the dataflow graph such that there is no path in the dataflow graph from the one or more identified nodes to any of the subset of nodes, and vice versa. Furthermore, the residual subgraph comprises a subset of the plurality of edges of the dataflow graph such that there is no path in the dataflow graph from the one or more identified nodes to any of the subset of edges, and vice versa. Using the residual subgraph for which there is no path in the dataflow graph from the one or more identified nodes to or from the one or more identified nodes defines one or more context values ​​that do not affect the execution of the one or more identified operations and are not affected by the results of the execution of the one or more identified operations, thus enabling correct restoration of values ​​required to resume execution of the thread after the execution of the one or more identified operations is completed. Furthermore, unlike a thread executing on a CPU, whose context comprises values ​​associated with predefined named general-purpose registers of the CPU, the thread context of a thread executing on a processing grid implementing a dataflow graph comprises values ​​that derive from the structure of the dataflow graph and not from the structure of the circuitry executing the thread.

[0041] Optionally, the one or more identified operations comprise a plurality of operations having inconsistent latencies.

[0042] Optionally, the context storage comprises a plurality of context entries, each context entry of the plurality of context entries for storing a plurality of runtime context values ​​of a thread context of one of the plurality of concurrently executing threads. A thread context may comprise more values ​​than fit in one context entry. Optionally, for at least one thread of the plurality of concurrently executing threads, a plurality of runtime context values ​​of a thread context of the at least one thread are stored in one or more context entries of the plurality of context entries. Optionally, the plurality of context entries are organized in a table having a plurality of rows, one row for each of the plurality of context entries. Optionally, the context storage is a reservation station.

[0043] Optionally, the context storage is additionally or alternatively used to store lookup tables implemented by nodes of a dataflow graph performed by multiple concurrently executing threads.

[0044] Optionally, the system comprises a plurality of context storages to enable individual scheduling of sequences of identified operations, where a first context storage is used to manage reuse of a first identified set of logic elements implementing one or more first identified operations of the thread, and a second context storage is used to manage reuse of a second identified set of logic elements implementing one or more second identified operations of the thread, with the advantage that more flexibility is provided in scheduling the execution of the multiple concurrently executing threads, further reducing the amount of time that processing resources of the processing grid are idle compared to scheduling the execution of the multiple concurrently executing threads while taking into account the one or more first identified operations together with the one or more second identified operations.

[0045] Additionally, in some of its embodiments, the present disclosure addresses another technical problem of managing the execution of a large number of concurrent threads that exceeds the amount of threads that can be executed simultaneously by a processing grid in a given processing grid configuration. In such embodiments, the multiple concurrent threads are a subset of a set of concurrent threads, each of the set of concurrent threads implementing a dataflow graph. At least some of the embodiments described herein improve the performance of a system by reducing the amount of context switching and scheduling overhead compared to standard methods for scheduling a large number of concurrent threads.

[0046] To that end, the present disclosure proposes, in some embodiments described herein, to store multiple other thread contexts in a buffer storage, one for each of a set of concurrently executing threads, and in each of multiple iterations, to retrieve other thread contexts of other threads of the set of concurrently executing threads from the buffer storage, and to use the other thread contexts when adding the other threads to the multiple threads for execution by the processing grid.

[0047] Optionally, the set of concurrently executing threads comprises one or more waiting threads that are not members of the multiple concurrently executing threads, the other threads being waiting threads for the one or more waiting threads. Optionally, the waiting thread is added to the multiple concurrently executing threads, and its execution is managed using the context storage, and optionally stores in the context storage the other thread context retrieved from the buffer storage. Optionally, the waiting thread is projected onto a processing grid without being added to the multiple concurrently executing threads, and optionally loads one or more values ​​of the other thread context into one or more other logic elements of the processing grid. Optionally, the waiting thread is selected in response to identifying that execution of at least one additional thread of the multiple concurrently executing threads has completed.

[0048] Optionally, a plurality of concurrently executing threads are selected from the set of concurrently executing threads after storing the contexts of the other plurality of threads in a buffer storage.

[0049] Optionally, the processing grid comprises a circuit (register circuit) for tracking one or more waiting threads. Optionally, the register circuit comprises a plurality of register entries, each for tracking at least one of the one or more waiting threads. In some embodiments described herein, the set of concurrently executing threads comprises one or more groups of waiting threads, the groups of waiting threads having a shared context. For such groups of waiting threads, the buffer storage may have a common thread context associated with each of the groups of waiting threads, and the register circuit may have a common register entry associated with each of the groups of waiting threads. When another thread is selected from the group of waiting threads, optionally, one or more private context values ​​of the other thread are generated and used when adding the other thread to the plurality of threads for execution by the processing grid.

[0050] To that end, the present disclosure proposes, in some embodiments described herein, to store multiple other thread contexts in a buffer storage, one for each of a set of concurrently executing threads, and in each of multiple iterations, to retrieve other thread contexts of other threads of the set of concurrently executing threads from the buffer storage, and to use the other thread contexts when adding the other threads to the multiple threads for execution by the processing grid.

[0051] Optionally, the system comprises a plurality of buffer storages to enable individual scheduling for the sets of multiple concurrently executing threads implementing the multiple data flow graphs, where a first buffer storage is used for managing the scheduling of the first set of concurrently executing threads and a second buffer storage is used for scheduling the second set of concurrently executing threads, with the advantage that the execution of one or more sets of concurrently executing threads can be more flexible in scheduling independently of each other, further reducing the time that processing resources of the processing grid are idle compared to scheduling the execution of one or more sets of concurrently executing threads as a single set of threads.

[0052] It should be noted that some embodiments of the present disclosure address both of the above technical problems, while some other embodiments address only one or the other.

[0053] Before describing at least one embodiment in detail, it is to be understood that the embodiments are not necessarily limited in their application to the details of construction and the arrangement of components and / or methods set forth in the following description and / or illustrated in the drawings and / or examples. The embodiments described herein are capable of other embodiments or of being practiced or carried out in various ways.

[0054] The embodiments may be systems, methods, and / or computer program products. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the embodiments.

[0055] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transitory signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a wave guide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through electrical wires.

[0056] The computer readable program instructions described herein can be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may comprise copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in each computing / processing device.

[0057] The computer readable program instructions for carrying out the operations of the embodiments may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Java, object-oriented Fortran, interpreted programming languages ​​such as JavaScript, Python, and traditional procedural programming languages ​​such as the "C" programming language, Fortran, etc., which may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or natively compiled or just-in-time (JIT) compiled source or object code. The computer readable program instructions may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), a coarse-grained reconfigurable architecture (CGRA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform aspects of the embodiments.

[0058] Aspects of the embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0059] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to generate a machine such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other apparatus to function in a particular manner such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions that perform aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0060] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable apparatus, or other device to execute a series of operational steps to generate a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0061] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions constituting one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, as well as combinations of multiple blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that executes the specified functions or operations, or executes a combination of special-purpose hardware and computer instructions.

[0062] Reference is now made to Figure 1, which illustrates a schematic block diagram of an exemplary dataflow graph 100, according to some embodiments. Optionally, dataflow graph 100 comprises a plurality of operations. Optionally, dataflow graph 100 comprises a plurality of nodes, for example, node 110A1, node 110A2, node 110A3, node 110A4, node 110A5, node 110A6, node 110A7, node 110A8, node 110B1, node 110B2, node 110B3, node 110B4, and one or more identified nodes 110C, collectively referred to as a plurality of nodes 110. Optionally, dataflow graph 100 comprises a plurality of edges, for example, edge 120A1, edge 120A2, edge 120A3, edge 120A4, edge 120A5, edge 120A6, edge 120A7, edge 120A8, edge 120A9, edge 120B1, edge 120B2, edge 120B3, edge 120B4, edge 120C1, and edge 120C2, collectively referred to as a plurality of edges 120. Optionally, each of the plurality of edges 120 is a directed edge that connects two of the plurality of nodes 110 and has a head node and a tail node. For example, node 110A2 is the tail node of edge 120B1, and node 110B1 is the head node of edge 120B1.

[0063] Optionally, the plurality of operations comprises one or more identified operations. Optionally, one or more nodes 110C perform the one or more identified operations. Optionally, the one or more identified operations have inconsistent latencies. Some examples of operations with inconsistent latencies include, but are not limited to, memory access operations, floating point mathematical operations, execution of other computational graphs, access to coprocessors, and access to peripheral devices connected to the processing grid. Examples of coprocessors include tensor core units and graphical processing units.

[0064] The context of the dataflow graph 100 for the one or more identified operations optionally comprises a value of a residual subgraph comprising a subset of nodes of the plurality of nodes 110 where no path exists in the dataflow graph 100 between any two of the one or more identified nodes 110C, and for each node of the subset of nodes, a path does not exist in the dataflow graph 100 between the node and the one or more identified nodes 110C. Optionally, the residual subgraph comprises a subset of edges of the plurality of edges 120, and for each edge of the subset of edges, a path does not exist in the dataflow graph 100 between the edge and the one or more identified nodes 110C. In this example, the residual subgraph 111 comprises a subset of nodes comprising nodes 110B1, 110B2, 110B3, and 110B4, collectively referred to as a subset of nodes 110B. In this example, no paths exist in dataflow graph 100 between one or more identified nodes 110C and any of nodes 110B1, 110B2, 110B3, and 110B4. Additionally, in this example, remaining subgraph 111 comprises a subset of edges, including edges 120C1 and 120C2, collectively referred to as the subset of edges 120C. In this example, no paths exist in dataflow graph 100 between one or more identified nodes 110C and any of edges 120C1 and 120C2.

[0065] Optionally, the runtime context values ​​of dataflow graph 100 are input values ​​of nodes of a plurality of nodes 120, such as the input value of node 110A2 or the input value of node 110B3. Optionally, the runtime context values ​​of dataflow graph 100 are input values ​​of nodes of residual graph 111, such as the input value of node 110B2 or the value of edge 120B2.

[0066] Optionally, the runtime context values ​​of dataflow graph 100 are output values ​​of nodes of multiple nodes 120, such as the output value of node 110B2 or the value of edge 120C1. Optionally, the runtime context values ​​of dataflow graph 100 are output values ​​of nodes of residual graph 111, such as the output value of node 110B3 or the value of edge 120B4. Optionally, the runtime context values ​​of dataflow graph 100 are output values ​​of one or more identified nodes 110C.

[0067] In one embodiment, the context of dataflow graph 100 for one or more identified operations comprises the value of edge 120B1, the value of edge 120B2, and the value of edge 120A7. In another embodiment, the context of dataflow graph 100 for one or more identified operations comprises the value of edge 120B3, the value of edge 120B4, and the value of edge 120A7. In another embodiment, the context of dataflow graph 100 for one or more identified operations comprises the value of edge 120B3, the value of edge 120C1, the value of edge 120C2, and the value of edge 120A7.

[0068] An entry node into a graph is a node that is not the head node of any edge of the graph. Optionally, the context of dataflow graph 100 for one or more identified operations comprises only the entry node into the remaining subgraph and the output values ​​of one or more identified nodes. In this example, such context comprises the value of edge 120B1 (or the input value of node 110B1), the value of edge 120B2 (or the input value of node 110B2), and the value of edge 120A7.

[0069] In some embodiments, the system executes multiple concurrent threads, each of which implements dataflow graph 100 .

[0070] Reference is now also made to Figure 2, which illustrates a schematic block diagram of an exemplary system 200 according to some embodiments. In such an embodiment, at least one hardware processor 201 is connected to at least one reconfigurable processing grid 210.

[0071] For the sake of brevity, hereafter the term "processing unit" will be used to mean "at least one hardware processor" and the terms are used interchangeably. The processing unit may be any programmable or non-programmable circuitry configured to perform the operations described below. The processing unit may comprise hardware as well as software. For example, the processing unit may comprise one or more processors and a transient or non-transient memory carrying programs that cause the processing unit to perform respective operations when the programs are executed by the one or more processors.

[0072] Further, hereinafter, for brevity, the term "processing grid" is used to mean "at least one reconfigurable processing grid", and the terms are used interchangeably. Optionally, the interconnected processing grid comprises a plurality of reconfigurable logic elements connected by a plurality of configurable data wiring branches. An example of a reconfigurable logic element is a processing circuit that can be operated to perform one of a set of specified mathematical and logical operations. Examples of mathematical operations include addition and subtraction. Examples of logical operations include a logical NOT operation and a logical AND operation. Optionally, the processing grid 210 comprises a plurality of logic elements consisting of logic element 211A, logic element 211B, logic element 211C, logic element 211D, and logic element 211E, collectively referred to as a plurality of logic elements 211. Optionally, the plurality of logic elements 211 are a plurality of reconfigurable logic elements. Optionally, the plurality of logic elements 211 are organized into a plurality of operation groups.

[0073] Optionally, processing grid 210 comprises one or more context storages, e.g., context storage 212 and additionally or alternatively other context storage 222. Optionally, context storage 212 is configured to store a plurality of thread contexts, each thread context for one of a plurality of concurrently executing threads. Optionally, each thread context of the plurality of thread contexts comprises, for that concurrently executing thread, one or more runtime context values ​​of a dataflow graph implemented by the concurrently executing thread. For example, each thread context may include a value of edge 120B1 for that thread. Optionally, context storage 212 is a reservation station. Optionally, other context storage 222 is a reservation station.

[0074] Reference is now also made to FIG. 3, which illustrates a schematic block diagram of an exemplary context storage 212, according to some embodiments. In such an embodiment, the context storage comprises a plurality of context entries, e.g., context entry 301A, context entry 301B, context entry 301C, and context entry 301D, collectively referred to as a plurality of context entries 301. Optionally, each of the plurality of context entries 301 is for storing a plurality of runtime context values ​​of one of a plurality of thread contexts. For example, context entry 301D may be for storing a first thread context of a first thread of a plurality of concurrently executing threads. Optionally, the plurality of runtime context values ​​of the first thread comprises runtime context value 310A, runtime context value 310B, runtime context value 310C, and runtime context value 310D, collectively referred to as a plurality of runtime context values ​​310. Optionally, context entry 301C is for storing a second thread context of a second thread of the plurality of concurrently executing threads. Optionally, a plurality of runtime context values ​​of the at least one thread context of the at least one thread are stored in a plurality of context entries of the plurality of context entries 301 .

[0075] Optionally, the plurality of context entries 301 are organized in a table. Optionally, the table has a plurality of rows. Optionally, each of the plurality of context entries 301 has a plurality of rows. Optionally, each row of the plurality of rows has a plurality of columns. Optionally, each of the plurality of runtime context values ​​310 is stored in one of the plurality of columns of the row for storing the context entry 301D.

[0076] Referring again to Figure 2, optionally, the context storage 212 is implemented in a memory area of ​​the processing grid 210. Optionally, the context storage 212 is implemented in one or more memory components 230 connected to the processing grid 210. Optionally, a portion of the context storage 212 is implemented in a memory area of ​​the processing grid 210, and another portion of the context storage 212 is implemented in the one or more memory components 230. Similarly, all or a portion of the other context storage 222 may be implemented in one or more of the other memory areas and one or more memory components 230 of the processing grid 210.

[0077] Optionally, the processing grid 210 comprises a dispatch circuit 213, optionally connected to the context storage 212. Optionally, the dispatch circuit 213 is used to calculate specific information of the completion of the context of the multiple contexts. A thread's context is completed when the processing of the thread can continue from a point where said processing was interrupted, for example waiting for the completion of an inconsistent latency operation. Optionally, the dispatch circuit 213 is used to configure one or more logic elements of the multiple logic elements 211 to execute one or more threads of the multiple concurrently executing threads, optionally by loading into the one or more logic elements one or more runtime context values ​​of the one or more threads retrieved from the context storage 212. Optionally, the dispatch circuit 213 is implemented in hardware. Optionally, the dispatch circuit 213 executes a set of dispatch computer instructions. Optionally, the set of dispatch computer instructions comprises a set of test instructions.

[0078] Similarly, the processing grid 210 is optionally connected to other context storage 222 and optionally comprises other dispatch circuitry 223 for managing the execution of multiple concurrently executing threads using the other context storage 222 .

[0079] Optionally, the processing grid 210 comprises one or more counters 240. Optionally, the multiple concurrently executing threads comprise a group of concurrently executing threads associated with one or more counters 240. Implementations exist where threads of the multiple concurrently executing threads can identify tasks to perform according to one or more counter values ​​of the one or more counters 240. For example, the multiple concurrently executing threads perform a loop, and one of the one or more counters 240 is used to count iterations of the loop. In another example, the multiple concurrently executing threads perform two or more nested loops, and each of the one or more counters counts iterations of one of the two or more nested loops. The one or more counter values ​​of the one or more counters 240 can indicate an iteration of the loop to be executed. Optionally, access to the one or more counters 240 comprises an atomic access consisting of reading the one or more counters 240 and incrementing or decrementing at least one of the one or more counters 240. Thus, after a first thread accesses the one or more counters 240, at least one of the one or more counters 240 is incremented or decremented such that each thread of multiple concurrently executing threads accessing the one or more counters 240 obtains a unique combination of the one or more counter values.

[0080] Optionally, the one or more counters 240 are a sequence of counters, e.g., when implementing two or more nested loops. In such an embodiment, a first counter of the one or more counters implements a counter of an inner loop of the two or more nested loops, and a second counter of the one or more counters 240 implements another counter of an outer loop of the two or more nested loops, successive to the first counter in the sequence of counters. Note that the designations "first counter" and "second counter" are intended to identify two successive counters in the sequence of counters, rather than referring to absolute ordinal identifiers in the sequence of counters.

[0081] Optionally, at least one of the one or more nested loops has a target amount of iterations, for example when the one or more counters 240 represent multiple dimensions of a set of tasks, such as an array of tasks. Optionally, incrementing the one or more counters 240 comprises incrementing a first count using modular arithmetic. Optionally, each of the one or more counters 240 has a modulus. Not all of the one or more counters 240 share a common modulus. Optionally, incrementing the one or more counters 240 comprises incrementing a second counter on condition that the first counter wraps after being incremented. Optionally, incrementing the one or more counters 240 includes incrementing a first counter of a sequence of counters, incrementing the second counter on condition that the first counter exceeds a maximum value, and setting the first counter to a new value calculated using the second counter. Additionally or alternatively, incrementing the one or more counters 240 may comprise setting a maximum value to another new value calculated using a second counter, e.g., when the inner loop is started from a value derived from a counter of the outer loop, e.g., each iteration of the outer loop comprises executing the inner loop a number of times derived using the current value of the second counter.

[0082] It should be noted that while the above description focuses on incrementing a counter, a first counter may be decremented and a second counter may be modified, provided that the first counter is lower than a minimum value. Optionally, the counters are incremented or decremented using a stride value different from 1.

[0083] In some embodiments, the multiple concurrently executing threads are a subset of the set of concurrently executing threads that each implement dataflow graph 100. Optionally, the set of concurrently executing threads comprises one or more waiting threads that are not members of the multiple concurrently executing threads and are pending execution.

[0084] Reference is now also made to FIG. 4 , which illustrates a schematic block diagram of another exemplary system 400 according to some embodiments. In such an embodiment, the processing grid 210 comprises a buffer storage 410. Optionally, the buffer storage 410 stores a further plurality of thread contexts for each of one or more threads of the set of concurrently executing threads. Optionally, the buffer storage 410 is implemented in yet another memory area of ​​the processing grid 210. Optionally, the buffer storage 410 is implemented in one or more memory components 230. Optionally, the buffer storage 410 comprises a plurality of buffer entries, each of the plurality of buffer entries for storing a thread context of at least one of one or more waiting threads of the set of concurrently executing threads.

[0085] Optionally, the processing grid 210 comprises a register circuit 411 for tracking the one or more waiting threads. Optionally, the register circuit 411 comprises a plurality of register entries, each for tracking at least one of the one or more waiting threads. Optionally, the processing grid 210 comprises a buffer dispatch circuit 412 for managing execution of a plurality of concurrently executing threads. Optionally, the buffer dispatch circuit 412 is connected to the buffer storage 410. Optionally, the buffer dispatch circuit 412 is connected to the register circuit 411. Optionally, the buffer dispatch circuit 412 is connected to the context storage 212, optionally via one or more ingress ports (not shown).

[0086] Optionally, processing grid 210 comprises one or more buffer storages. Optionally, one or more other buffer storages store further thread contexts, each for one or more of one or more other sets of concurrently executing threads. Similarly, processing grid 210 optionally comprises one or more other buffer dispatch circuits connected to the one or more other buffer storages, and one or more other register circuits connected to the one or more other buffer dispatch circuits, additionally or alternatively.

[0087] In some embodiments, processing unit 201 executes multiple concurrent threads and additionally or alternatively configures processing grid 210 to execute a set of concurrent threads.

[0088] Reference is now also made to Fig. 5, which illustrates a flow chart that generally illustrates an optional computational flow 500, according to some embodiments. In such an embodiment, at 510, the processing unit 201 identifies a residual sub-graph 111. Optionally, the processing unit 201 uses the residual sub-graph 111 to determine a number of runtime context values ​​for each of a number of concurrently executing threads, as described above in Fig. 1.

[0089] Optionally, at 520, the processing unit 201 configures the processing grid 210. Optionally, configuring the processing grid 210 comprises operating one or more of the plurality of logic elements 211. Optionally, configuring the processing grid 210 comprises, but is not limited to, operating one or more of the context storage 212, the dispatch circuitry 213, the buffer storage 410, the register 411, and the buffer dispatch circuitry 412. Optionally, configuring the processing grid 210 comprises loading one or more values ​​into one or more of the context storage 212 and the buffer storage 410.

[0090] In some embodiments, the processing unit 201 configures the processing grid 210 to perform one or more methods. Reference is now also made to FIG. 6, which shows a flow chart that generally illustrates an optional computational flow 600 for a processing grid, according to some embodiments. Optionally, the processing unit 201 configures the processing grid 210 to perform the method 600. In such an embodiment, in 601, the processing grid 210 executes a first thread of a plurality of concurrently executing threads using at least some of the plurality of logic elements 211. Optionally, one or more identified operations of the dataflow graph 100 are executed by one or more identified logic elements of the plurality of logic elements 211, for example, the identified logic element 211C. Optionally, in case the plurality of logic elements 211 are organized into a plurality of computational groups, the one or more identified logic elements are a subset of the plurality of computational groups. At 610, the processing grid 201 optionally stores one or more runtime context values ​​of the first thread, e.g., runtime context values ​​comprising runtime context value 310A, in the context storage 212, e.g., in context entry 301D. When the multiple context entries 310 are organized in a table, with context entry 301D being a row of the table, the processing grid 201 optionally stores the runtime context value 310A in one of the multiple columns of the row. Optionally, the runtime context value 310A is an input value of an entry node to the residual subgraph 111, e.g., the input value of node 110B1. Optionally, when the context entry 301D is a row of a table, the processing grid 210 stores the runtime context value 310A in one or more identified columns of the context storage 212. Optionally, the processing grid 201 stores the runtime context value 310A while waiting for the identified logic element 211C to complete execution of the one or more identified operations of the first thread.Further, while waiting for completion of execution of the one or more identified operations of the first thread by the identified logic element 211C, at 612 the processing grid 210 optionally configures at least some logic elements to execute a second thread of the multiple concurrently executing threads such that the processing grid 210 executes the one or more identified operations of the second thread by the identified logic element 211C.

[0091] Optionally, upon completion of execution of the one or more identified operations of the first thread, at 615, the processing grid 210 optionally stores one or more result values ​​in the context entry 301D. Optionally, the one or more result values ​​are a result of executing the one or more identified operations. For example, if the one or more identified operations comprise a memory access, the one or more result values ​​may include a data value obtained by the memory access.

[0092] In 620, the processing grid 210 optionally calculates a specific information that the context entry 301D is completed. Optionally, the specific information that the context entry 301D is completed is calculated according to a result of applying one or more tests to the plurality of runtime context values ​​310. Optionally, the one or more tests are applied by the dispatch circuitry 213. Optionally, the dispatch circuitry executes a sequence of test instructions to apply the one or more tests. Optionally, applying the one or more tests comprises checking whether the plurality of runtime context values ​​310 are valid. According to some embodiments, to that end, the context entry 301D comprises a plurality of validity bits. Optionally, each of the plurality of validity bits is associated with one of the plurality of runtime context values ​​310. Optionally, applying the one or more tests comprises applying a specified bitwise mask to the plurality of validity bits, the specified bitwise mask indicating which of the plurality of runtime context values ​​310 need to be valid for the context entry 301D to be completed. In some embodiments, the specified bitwise mask is applied to a number of validity bits that describe a row of a table of context entries.

[0093] Optionally, the processing grid 210 selects the first thread for execution, at 630. Optionally, the processing grid 210 selects the first thread using the dispatch circuitry 213.

[0094] Multiple threads of the multiple concurrently executing threads may complete execution of the respective one or more identified operations. Optionally, the first thread is selected according to a dispatch policy. For example, the first thread may be the first thread found. Optionally, the dispatch policy comprises using round robin to prevent thread starvation. Optionally, the dispatch policy is based on a timestamp, for example dispatching the earliest or latest completed thread. Optionally, the dispatch policy is based on a priority value. Optionally, the priority value is assigned to the thread. Optionally, the priority value is calculated by the thread itself. Optionally, the priority value determines a partial ordering of priorities among the multiple concurrently executing threads.

[0095] Optionally, at 622, the processing grid 210 identifies that the plurality of threads have completed execution of the respective one or more identified operations by calculating other identifying information that one or more other context entries of the plurality of context entries 301 have completed. Optionally, the other identifying information is calculated according to one or more other results of applying one or more tests to one or more other runtime context values ​​of the one or more other context entries. Optionally, the processing grid performs 622 before selecting the first thread at 630, for example when using a priority-based dispatch policy.

[0096] At 632, the processing grid 210 optionally retrieves the runtime context value 310A from the context storage 212. Optionally, the processing grid 210 executes 632, conditional on the context entry 301D being completed. Optionally, the processing grid 210 retrieves the runtime context value 310A from the context storage 212 using the dispatch circuitry 212. Optionally, at 640, the processing grid 210 resumes execution of the first thread by executing one or more other operations of the plurality of operations of the first thread, e.g., one or more operations performed in node 110A6. Optionally, the processing grid loads the retrieved runtime context value 310A into one or more of the plurality of logic elements 211. Optionally, the processing grid 210 executes 640, conditional on the context entry 301D being completed.

[0097] In some embodiments, it may be necessary to suspend the execution of the first thread. Some examples include a breakpoint set for debugging. Optionally, the breakpoint is conditioned based on one or more values ​​of the plurality of runtime context values ​​311. Upon pausing the execution of the first thread, the processing grid 210 may stop the execution of the first thread. Additionally or alternatively, the processing grid 210 may provide context information about the first thread to one or more other software objects (some examples are a debugger and a logger) executing in the system 200. Optionally, conditional on the mark added to the context entry 301D, the processing grid 210 denies the execution of one or more other operations of the first thread. Additionally or alternatively, conditional on the mark, the processing grid 210 optionally provides one or more of the plurality of runtime context values ​​310 to one or more other software objects. Optionally, the one or more other software objects are implemented in another part of the processing grid 210. Optionally, one or more other software objects are executed by processing unit 201. Optionally, one or more other software objects are executed by other processing units (not shown) connected to processing grid 210.

[0098] In some embodiments, the context entry may be reused. For example, if multiple context entries 301 are organized in a table, one of multiple columns of a row of the table may be reused. In such an embodiment of 650, the processing grid 210 optionally stores one or more other runtime context values ​​of the first thread in a column used to store one or more runtime context values ​​comprising runtime context value 310A in 610. The context entry may be reused when executing another identified operation of the first thread having an inconsistent latency. Optionally, the context entry is reused while waiting for another execution of one or more identified operations of the first thread to complete. Optionally, the first thread is executed in each of the multiple iterations. Optionally, the processing grid 210 executes 610 in one of the multiple iterations and executes 650 in another of the multiple iterations.

[0099] In some embodiments, one or more nodes of the dataflow graph 100 implement a lookup table. When the multiple context entries 301 are organized in a table, configuring the processing grid 210 for execution of the first thread optionally comprises storing a lookup table in one or more other columns of the multiple columns of the table other than the one or more columns for storing the runtime context values ​​310A. If the lookup table is stored in the one or more other columns, the one or more other columns may be read but not written to by any of the multiple concurrently executing threads.

[0100] Optionally, the thread accesses the lookup table using an instruction comprising an index and a value of one or more other columns. Optionally, if the index identifies a column that is part of the lookup table, the value is used as an indication of which row to access.

[0101] Optionally, the processing grid 210 is further configured by the processing unit 201 to execute the first thread in each of the multiple thread iterations. In some embodiments, one or more additional context entries of the multiple context entries 301 are used to provide at least a portion of the context of the first thread that is common to the multiple thread iterations each executing the first thread. Optionally, the one or more additional context entries comprise one or more additional context values ​​that are an entry value into the residual subgraph, such as, for example, an input value of node 110B2. Optionally, the one or more other context entries provide other portions of the context for the multiple thread iterations for executing the thread and comprise one or more further context values ​​that are internal to the residual subgraph and may be different for some of the multiple thread iterations, such as, for example, a value of edge 120C2. Optionally, the processing grid 210 is configured to retrieve at least one of the one or more additional context values ​​from the context storage 212, optionally for a next thread iteration for executing the first thread, upon completion of execution of the one or more identified operations of the first thread. Optionally, at least one of the one or more additional context values ​​is loaded into one or more of the plurality of logic elements 211 before the other values ​​are loaded.

[0102] In some embodiments with multiple context storages, the processing unit 201 configures the processing grid 210 to perform any of the following methods.

[0103] Reference is now also made to FIG. 7, which illustrates a flow chart that generally depicts another optional flow of operations 700 for a processing grid, according to some embodiments. In such an embodiment, upon completion of execution of the one or more identified operations of the first thread, at 710 the processing grid 210 stores one or more further runtime context values ​​of the first thread in the other context storage 222. Optionally, at 720 the processing grid 210 further executes the first thread, and at 730 the processing grid 210 further executes the first thread while optionally executing one or more other identified operations of a second thread by one or more other identified logic elements, e.g. logic element 211E, while waiting for completion of execution of the one or more other identified operations of the first thread by logic element 211E. Optionally, the processing grid 210 uses the other context storage 222 and the other dispatch circuit 223 to manage the execution of multiple concurrently executing threads.

[0104] As mentioned above, in some embodiments described herein, the multiple concurrently executing threads are a subset of the set of concurrently executing threads that each implement dataflow graph 100. In such embodiments, the set of concurrently executing threads comprises one or more waiting threads that are not members of the multiple concurrently executing threads and are pending execution. Optionally, processing grid 210 manages the execution of the set of concurrently executing threads using buffer storage 410. Optionally, to manage the execution of the set of concurrently executing threads, processing unit 201 configures processing grid 210 to perform the following method:

[0105] Reference is now also made to FIG. 8, which illustrates a flow chart that generally illustrates yet another optional flow of operations 800 for a processing grid, according to some embodiments. Optionally, at 810, the processing circuitry 210 stores in the buffer storage 410 one or more additional runtime context values ​​of one or more waiting threads. To increase utilization of the plurality of logic elements while waiting for completion of execution of one or more identified operations of the set of concurrently executing threads, it may be beneficial to start execution of the set of concurrently executing threads only if there are sufficient waiting threads. Thus, at 805, the processing grid 210 optionally stalls execution of at least a portion of the set of concurrently executing threads until the amount of one or more waiting threads exceeds a threshold value. Optionally, the processing grid 210 uses the register circuitry 411 to stall execution of at least a portion of the set of concurrently executing threads.

[0106] Optionally, in each of the multiple iterations, at 815, the processing grid 210 identifies that one or more additional threads of the multiple concurrently executing threads have completed execution. Optionally, at 820, the processing grid 210 selects at least one waiting thread from the register circuitry 411, and at 822, the processing grid 210 optionally retrieves one or more additional runtime context values ​​of the at least one waiting thread from the buffer storage 410. Optionally, the processing grid 210 retrieves the one or more additional runtime context values ​​using the buffer dispatch circuitry 412.

[0107] Optionally, the processing grid 210 adds the completed one or more additional threads to one or more other buffer storages.

[0108] At 830, the processing grid 210 optionally adds at least one waiting thread to the plurality of concurrently executing threads for execution by the plurality of logic elements 211. Optionally, adding the at least one waiting thread to the plurality of concurrently executing threads comprises storing one or more additional runtime context values ​​of the at least one waiting thread in the context storage 212. Storing the one or more additional runtime context values ​​of the at least one waiting thread in the context storage 212 facilitates execution of the at least one waiting thread in 612 of method 600 above.

[0109] Optionally, at 830, the processing grid 210 loads one or more additional runtime context values ​​of the at least one waiting thread into one or more of the plurality of logic elements and begins execution of the at least one waiting thread without using the context storage 212.

[0110] Optionally, at 821, the processing grid 210 associates each waiting thread of the at least one waiting thread (which is added to the plurality of concurrently executing threads at 830) with a context identification value. Optionally, the context identification value indicates a context of the waiting thread in the context storage 212. For example, the context identification value may be a thread identification value of the waiting thread.

[0111] In some embodiments, each group of waiting threads of the one or more groups of waiting threads may be managed using some common resources. In such embodiments, the buffer dispatch 412 is optionally configured to generate, for at least one group of the one or more groups of waiting threads, a common thread context in the buffer storage 410 associated with each of the at least one group of waiting threads. Optionally, the buffer dispatch 412 is configured to generate a common register entry in the register circuit 411 associated with each of the at least one group of waiting threads. Optionally, when selecting a new thread of the at least one group of waiting threads as the at least one waiting thread in 820, the buffer dispatch 412 is optionally configured to calculate one or more private context values ​​of the new thread. Optionally, the one or more private context values ​​are derived from a row index in the buffer storage 410 or other row index in the context storage 212. Optionally, the one or more private context values ​​are derived from a column index in the buffer storage 410 or other column index in the context storage 212. Optionally, the private value is a running counter, and is incremented or decremented sequentially using a step value, optionally using modulo arithmetic. Optionally, the private value is calculated using a hash function. Optionally, the private value is calculated using a function for locality of the data structure. Optionally, the private value is a specified value.

[0112] The description of various embodiments has been presented for the purpose of explanation, but is not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0113] It is anticipated that many related processing grids will be developed during the life of the patent starting with this application, and the scope of the term processing grid is intended to include, a priori, all such new technologies.

[0114] As used herein, the term "about" refers to ±10%.

[0115] The terms "comprises," "comprising," "including," "including," "having," and their conjugations mean "including, but not limited to." This term encompasses the terms "consisting of" and "consisting essentially of."

[0116] "Consisting essentially of" means that a composition or method may include additional components and / or steps, but only if the additional components and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0117] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" can include a plurality of compounds, including mixtures thereof.

[0118] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." An embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments.

[0119] As used herein, the word "optionally" means "is provided in some embodiments and is not provided in other embodiments." Any particular embodiment may include multiple "optional" features, unless such features are inconsistent.

[0120] Throughout this application, various embodiments may be described in a range format. It should be understood that the description in range format is merely for convenience and brevity, and should not be construed as an inflexible limitation on the scope of the embodiments. Thus, the description of a range should be considered to have specifically disclosed all possible subranges as well as individual numerical values ​​within that range. For example, description of a range such as 1 to 6 should be considered to have specifically disclosed subranges such as 1 to 3, 1 to 4, 1 to 5, 2 to 4, 2 to 6, 3 to 6, and individual numerical values ​​within that range, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0121] Whenever a numerical range is given herein, it is meant to include any cited numbers (fractional or integer) within the given range. As used herein, the phrases "ranging between" a first indicated numerical value and a second indicated numerical value, and "ranging between" a first indicated numerical value and a second indicated numerical value, are used interchangeably and are meant to include the first and second indicated numerical values ​​and all fractions and integers therebetween.

[0122] It is understood that certain features of embodiments that are described in the context of separate embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of an embodiment that are described in the context of a single embodiment for brevity may also be provided separately or in any suitable subcombination or in any other described embodiment, as appropriate. Certain features described in the context of various embodiments are not considered essential features of those embodiments, unless the embodiments are inoperable without those elements.

[0123] While the embodiment has been described in conjunction with a specific embodiment thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications, and variations that fall within the spirit and broad scope of the appended claims.

[0124] It is the intention of the applicants that all publications, patents, and patent applications referenced herein are incorporated herein by reference in their entirety as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated herein by reference. Furthermore, citation or identification of a reference in this application should not be construed as an admission that the reference is available as prior art to the present invention. To the extent section headings are used, they should not be construed as necessarily limiting. Additionally, the priority documents of this application are incorporated herein by reference in their entirety.

Claims

1. 1. A system for processing multiple concurrent threads, comprising: at least one reconfigurable processing grid, A plurality of logic elements; a context storage configured to store a plurality of thread contexts, each thread context for one of a plurality of concurrently executing threads, each concurrently executing thread implementing a dataflow graph comprising a plurality of operations including at least one identified operation, each of the plurality of thread contexts comprising, for that concurrently executing thread, at least one runtime context value of the dataflow graph implemented by that concurrently executing thread; at least one reconfigurable processing grid comprising: At least one hardware processor, the at least one reconfigurable processing grid comprising: executing a first thread of the plurality of concurrently executing threads; During execution of the first thread, storing the at least one runtime context value of the first thread in the context storage; executing, by at least one identified logic element of the plurality of logic elements, at least one identified operation of a second thread of the plurality of concurrently executing threads while waiting for completion of execution of at least one identified operation of a plurality of operations of a first thread by the at least one identified logic element of the plurality of logic elements; when execution of the at least one identified operation of the first thread is completed, obtaining the at least one runtime context value for the first thread from the context storage; executing at least one other operation of the plurality of operations of the first thread; at least one hardware processor configured to A system comprising:

2. 2. The system of claim 1, wherein the context storage comprises a plurality of context entries, each of the plurality of context entries for storing a plurality of runtime context values ​​of one of the plurality of thread contexts.

3. 3. The system of claim 2, wherein for at least one thread of the plurality of concurrently executing threads, the plurality of runtime context values ​​of the at least one thread are stored in a plurality of context entries of the plurality of context entries.

4. the plurality of context entries are organized in a table having a plurality of rows, one row for each of the plurality of context entries; 4. The system of claim 3, wherein each row of the plurality of rows has a plurality of columns such that each of the plurality of runtime context values ​​of a thread context stored in the row is stored in one of the plurality of columns.

5. the data flow graph comprises a plurality of nodes and a plurality of edges; At least one node of the plurality of nodes implements a lookup table; 5. The system of claim 4, wherein configuring the at least one reconfigurable processing grid to execute the first thread comprises storing the lookup table in at least one other column of the plurality of columns.

6. The at least one reconfigurable processing grid comprises: when execution of the at least one identified operation of the first thread is completed, storing at least one result value resulting from execution of the at least one identified operation of the first thread in a context entry of the plurality of context entries for storing at least a portion of the thread context of the first thread; The system of claim 2 further configured to:

7. a first context entry of the plurality of context entries stores a plurality of runtime context values ​​for the first thread; The at least one reconfigurable processing grid comprises: calculating a specific indication that the first context is completed according to an outcome of at least one test applied to the plurality of runtime context values ​​of the first context entry; obtaining the at least one runtime context value of the first thread and executing the at least one other operation conditional on specific information that the first context has completed; The system of claim 2 , configured to:

8. The system of claim 7 , wherein the at least one reconfigurable processing grid further comprises dispatch circuitry for applying the at least one test to the plurality of runtime context values.

9. 9. The system of claim 8, wherein applying the at least one test to the plurality of runtime context values ​​comprises the dispatch circuitry executing a set of test instructions.

10. 8. The system of claim 7, wherein the at least one reconfigurable processing grid is further configured to select the first thread for executing the at least one other operation of the plurality of operations according to a dispatch policy.

11. The at least one reconfigurable processing grid comprises: conditional on a mark added to one or more context entries of the plurality of context entries for storing at least a portion of the thread context of the first thread; refusing to execute the at least one other operation of the plurality of operations of the first thread; and providing at least one of the plurality of thread context values ​​of the first thread to at least one other software object; The system of claim 7 , further configured to perform at least one of the following:

12. the data flow graph comprises a plurality of nodes and a plurality of edges; the at least one identified operation is represented in the dataflow graph by at least one identified node of the plurality of nodes; the at least one hardware processor is further configured to identify a subgraph (a residual subgraph) in the dataflow graph consisting of a subset of nodes of the plurality of nodes and a subset of edges of the plurality of edges, whereby no path exists in the dataflow graph between any two of the at least one identified nodes, and for each node of the subset of nodes, no path exists in the dataflow graph between the at least one identified node and the node, and for each edge of the subset of edges, no path exists in the dataflow graph between the at least one identified node and the edge, the at least one runtime context value is at least one edge value of at least one of the subset of edges. The system of claim 1 .

13. the data flow graph is a directed graph; each of the plurality of edges has a head node of the plurality of nodes and a tail node of the plurality of nodes; the subset of nodes comprises one or more entry nodes, each of which is an entry node of the residual subgraph, and the entry node is not a head node of any of the subset of edges; the at least one runtime context value is at least one input value of at least one of the one or more entry nodes. The system of claim 12.

14. 2. The system of claim 1 , wherein the at least one identified operation comprises at least one of a memory access operation, a floating point math operation, execution of other computational graphs, access to a co-processor, and access to a peripheral device connected to the at least one reconfigurable processing grid.

15. the plurality of concurrently executing threads is a subset of a set of concurrently executing threads, each of the set of concurrently executing threads implementing a dataflow graph; the system further comprising a buffer storage for storing a plurality of other thread contexts, each for at least one of the set of concurrently executing threads; the at least one hardware processor further configuring the at least one reconfigurable processing grid by: storing in the buffer storage one or more additional runtime context values ​​of one or more waiting threads, the one or more waiting threads being not members of the plurality of concurrently executing threads; In each of the plurality of iterations, determining that at least one additional thread of the plurality of concurrently executing threads has completed execution; retrieving, for at least one of the one or more waiting threads, the at least one additional runtime context value from the buffer storage; adding the at least one waiting thread to a plurality of concurrently executing threads for execution by the plurality of logic elements; and further configured to: The system of claim 1 .

16. The at least one reconfigurable processing grid comprises: a register circuit for tracking the one or more waiting threads; additional dispatch circuitry for managing execution of said multiple concurrently executing threads; Further equipped with The additional dispatch circuitry includes: selecting the at least one waiting thread from the register circuit; retrieving the at least one additional runtime context value of the at least one waiting thread from the buffer storage; adding said at least one waiting thread to said plurality of concurrently executing threads for execution by said plurality of logic elements; The system of claim 15 configured to:

17. adding the at least one waiting thread to the plurality of concurrently executing threads comprises storing the at least one additional runtime context value of the at least one waiting thread in the context storage.

17. The system of claim 16.

18. The additional dispatch circuitry includes: associating with each of the at least one waiting thread a context identification value indicating a thread context of the waiting thread in the context storage; The system of claim 16 further configured to:

19. the buffer storage comprises a plurality of buffer entries, each for storing a thread context of at least one of the one or more waiting threads; the register circuitry comprises a plurality of register entries, each for tracking at least one of the one or more waiting threads; The additional dispatch circuitry includes: generating, for at least one group of waiting threads among the one or more waiting threads, a common thread context in the buffer storage associated with each of the group of at least one waiting thread; creating a common register entry in a register circuit associated with each of said at least one group of waiting threads; when selecting a new thread from the register circuitry of the group of the at least one waiting thread as the at least one waiting thread, calculating at least one private context value of the new thread; The system of claim 16 , further configured to:

20. 17. The system of claim 16, wherein the registerer circuitry is further configured to stall execution of at least a portion of the set of concurrently executing threads until an amount of the one or more waiting threads exceeds a threshold.

21. the plurality of logic elements being a plurality of reconfigurable logic elements organized into a plurality of computational groups; The system of claim 1 , wherein the at least one identified logic element is a subset of the plurality of computational groups.

22. The system of claim 1 , wherein the runtime context values ​​of the dataflow graph are node input or output values ​​of a plurality of nodes of the dataflow graph.

23. the at least one hardware processor is further configured to configure the at least one reconfigurable processing grid to execute the first thread in each of a plurality of thread iterations; the context storage comprises at least one additional context entry for storing additional runtime context values ​​of the dataflow graph, the additional runtime context values ​​being common to the multiple thread iterations; and upon completion of execution of the at least one identified operation of the first thread, the reconfigurable processing grid is further configured to retrieve at least one of the additional plurality of runtime context values ​​from the context storage. The system of claim 1 .

24. the at least one reconfigurable processing grid further comprises at least one other context storage; the at least one hardware processor for configuring the at least one reconfigurable processing grid: when execution of the at least one identified operation of the first thread is completed, storing at least one additional runtime context value of the first thread in the at least one other context storage; further executing the first thread; During further execution of the first thread, executing, by the at least one other identified logic element, the at least one other identified operation of another second thread of the plurality of threads while waiting for completion of further execution of the at least one other identified operation of the plurality of operations of the first thread by the at least one other identified logic element of the plurality of logic elements. further comprising: The system of claim 1 .

25. the at least one reconfigurable processing grid further comprises at least one counter; the plurality of concurrently executing threads comprises a group of concurrently executing threads associated with the at least one counter; The system of claim 1 , wherein the at least one runtime context value comprises at least one counter value that is read from the at least one counter by accessing the at least one counter.

26. 26. The system of claim 25, wherein each of the group of concurrently executing threads implements a specified dataflow graph.

27. 26. The system of claim 25, wherein accessing the at least one counter comprises an atomic access comprising reading the at least one counter and incrementing the at least one counter.

28. the at least one counter is a sequence of counters; Incrementing the at least one counter includes: incrementing a first counter in the sequence of counters using modular arithmetic and, subject to the first counter wrapping around after being incremented, incrementing a second counter successive to the first counter in the sequence of counters; incrementing a first counter of the sequence of counters, if the first counter exceeds a maximum value; incrementing a second counter subsequent to the first counter in the sequence of counters; setting the first counter to a new value calculated using the second counter; and setting said maximum value to another new value calculated using said second counter; incrementing a first counter; and 28. The system of claim 27, comprising at least one of:

29. 1. A method for processing multiple concurrent threads, comprising: executing a first thread of the plurality of concurrently executing threads, each concurrently executing thread implementing a data flow graph comprising a plurality of operations comprising at least one identified operation; During execution of the first thread, storing at least one runtime context value of the dataflow graph performed by the first thread in a context storage, the context storage configured to store a plurality of thread contexts, each thread context for one of the plurality of concurrently executing threads, each of the plurality of thread contexts comprising, for that concurrently executing thread, the at least one runtime context value of the dataflow graph performed by the concurrently executing thread; executing, by the at least one identified logic element, the at least one identified operation of a second thread of the plurality of concurrently executing threads while waiting for completion of execution of the at least one identified operation of the plurality of operations of the first thread by the at least one identified logic element of a plurality of logic elements; when execution of the at least one identified operation of the first thread is completed, obtaining the at least one runtime context value for the first thread from the context storage; executing at least one other operation of the plurality of operations of the first thread; and A method comprising:

30. 1. A software program product for executing multiple concurrent threads of execution, comprising: a non-transitory computer readable storage medium; first program instructions for executing a first thread of a plurality of concurrently executing threads, each concurrently executing thread implementing a dataflow graph comprising a plurality of operations comprising at least one identified operation; Second program instructions, during execution of the first thread, storing at least one runtime context value of the dataflow graph performed by the first thread in a context storage, the context storage configured to store a plurality of thread contexts, each thread context for one of the plurality of concurrently executing threads, each of the plurality of thread contexts comprising, for that concurrently executing thread, at least one runtime context value of the dataflow graph performed by that concurrently executing thread; executing, by the at least one identified logic element, the at least one identified operation of a second thread of the plurality of concurrently executing threads while waiting for completion of execution of the at least one identified operation of the plurality of operations of the first thread by the at least one identified logic element of a plurality of logic elements; when execution of the at least one identified operation of the first thread is completed, obtaining the at least one runtime context value for the first thread from the context storage; executing at least one other operation of the plurality of operations of the first thread; and and second program instructions for performing Equipped with the first and second program instructions are executed by at least one computerized processor from the non-transitory computer readable storage medium. Software program products.

Images