Executing concurrent threads on reconfigurable processing grid

Abstract

A system for processing a plurality of concurrent threads includes a reconfigurable processing grid comprising logic elements and context storage means for storing thread contexts, each thread context corresponding to one of the plurality of concurrent threads, each concurrent thread implementing a dataflow graph comprising identified operations; and a hardware processor configured to configure the reconfigurable processing grid: execute a first thread of the plurality of concurrent threads; and while executing the first thread: storing a runtime context value of the first thread in a context storage; when waiting for the identified logic element to complete the identified operation, executing the identified operation of the second thread by the identified logic element; and when execution of the identified operation of the first thread is completed, retrieving a runtime context value of the first thread from the context storage, and executing another operation of the first thread.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Patent Application No. 18 / 218,152 filed on July 5, 2023 and U.S. Patent Application No. 18 / 409,869 filed on July 5, 2023, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] Some embodiments described in this disclosure relate to a reconfigurable processing grid and, more particularly, but not exclusively, to executing one or more dataflow graphs on a reconfigurable processing grid. Background Art

[0004] As used herein, the term "reconfigurable processing grid" refers to a processing circuit comprising a plurality of reconfigurable logic elements connected by a plurality of reconfigurable data routing nodes, wherein, in each iteration of one or more iterations, the plurality of reconfigurable logic elements and, additionally or alternatively, the plurality of reconfigurable data routing nodes can be manipulated to perform one or more operations. As used herein, the term data flow represents a computer programming paradigm that models at least a portion of a software program as a directed graph (data flow graph) of data flowing between operations, so that a series of operations are applied to each data element in a sequence of data elements of the data flow graph. Optionally, the data flow graph includes a plurality of nodes and a plurality of directed edges, each node applying an operation to a data element, each directed edge connecting two of the plurality of nodes and indicating the data flow between the two nodes. In the field of computer science, an execution thread is a sequence of computer instructions that can be independently managed by a scheduler. For the sake of brevity, the term "thread" is used to represent an "execution thread", and these terms are used interchangeably herein. A thread can implement a data flow graph. As used herein, the term "projection" refers to the process of manipulating one or more reconfigurable logic elements of a reconfigurable processing grid and, additionally or alternatively, manipulating one or more reconfigurable data routing nodes of the reconfigurable processing grid to execute a data flow graph. Thus, projecting a thread implementing a data flow graph onto a reconfigurable processing grid refers to configuring the reconfigurable processing grid by manipulating one or more reconfigurable logic elements of the reconfigurable processing grid and, additionally or alternatively, manipulating one or more reconfigurable data routing nodes of the reconfigurable processing grid to execute the data flow graph implemented by the thread.

[0005] In the field of computer science, concurrent computing refers to the simultaneous execution of multiple threads of execution of a software program. The simultaneous execution of multiple threads of a software program allows for improved overall system performance and responsiveness. Metrics used to measure system performance include, but are not limited to: the amount of tasks a system executes in a specified amount of time (throughput), the amount of time it takes to complete task execution (latency), and the amount of computer memory used by the system while it is running. Concurrent computing can be used to improve throughput and reduce latency in a system.

[0006] It may be that each of the plurality of concurrent threads includes one or more identified operations. When the plurality of concurrent threads are executed simultaneously on the reconfigurable processing grid, each of the plurality of concurrent threads is projected onto a portion of the reconfigurable processing grid, that is, some of the plurality of logic elements of the reconfigurable processing grid are manipulated to execute the concurrent thread, for example, to execute a data flow graph implemented by the concurrent thread.

[0007] There are situations where the delay in the completion of computer instructions is inconsistent. When executed more than once, such operations may require different amounts of execution time and may additionally or alternatively require more time to complete than other instructions. Some examples of such inconsistent delay operations include, but are not limited to, memory access, access to peripheral devices, and execution of compute kernels. Summary of the invention

[0008] Some embodiments described in the present disclosure are directed to a system and method for executing multiple concurrent threads by storing multiple thread contexts (each thread context is for one of the multiple concurrent threads) in a context storage device and managing the execution of the multiple concurrent threads on a reconfigurable processing grid using the context storage device, wherein each thread context is for one of the multiple concurrent threads. Optionally, one or more logic elements manipulated to perform an identified operation of a first thread are used to perform an identified operation of a second thread while the first thread is waiting for completion of the identified operation of the first thread without reconfiguring the one or more logic elements to perform the identified operation of the second thread. Optionally, the identified operation has an inconsistent delay.

[0009] The above-mentioned and other objects are achieved by the features of the independent claims. Further implementation forms are evident from the dependent claims, the description and the drawings.

[0010] According to a first aspect, a system for processing multiple concurrent threads comprises: at least one reconfigurable processing grid including multiple logic elements and a context storage device, the context storage device being configured to store multiple thread contexts, each thread context for one of multiple concurrent threads, each concurrent thread implementing a data flow graph including multiple operations, the multiple operations including at least one identified operation, wherein each thread context of the multiple thread contexts for a concurrent thread of the multiple concurrent threads comprises at least one runtime context value of the data flow graph implemented by the concurrent thread; and at least one hardware processor configured to configure the at least one reconfigurable processing grid to perform the following operations: execute a first thread of the multiple concurrent threads and store the at least one runtime context value of the first thread in the context storage device while executing the first thread; while waiting for at least one identified logic element of the multiple logic elements to complete execution of at least one identified operation of the multiple operations of the first thread, the at least one identified logic element executes at least one identified operation of a second thread of the multiple concurrent threads; and when execution of the at least one identified operation of the first thread is completed, retrieve the at least one runtime context value of the first thread from the context storage device; and execute at least one other operation of the multiple operations of the first thread. Storing the thread context for each of the plurality of concurrent threads in the context storage device enables pausing and resuming execution of one or more of the plurality of concurrent threads without manipulating the processing grid, thereby reducing the complexity of reducing the amount of time that one or more identified logic elements are idle while waiting for the execution of one or more operations of the first thread to complete. This is beneficial in: improving the throughput of the system and reducing the system latency in executing one or more tasks of the system.

[0011] According to a second aspect, a method for processing multiple concurrent threads includes: executing a first thread among the multiple concurrent threads, each concurrent thread implementing a data flow graph including multiple operations, the multiple operations including at least one identified operation; and while executing the first thread, storing at least one runtime context value of the data flow graph implemented by the first thread in a context storage device, wherein the context storage device is configured to store multiple thread contexts, each thread context for one of the multiple concurrent threads, wherein, for a concurrent thread among the multiple concurrent threads, each of the multiple thread contexts includes at least one runtime context value of the data flow graph implemented by the concurrent thread; while waiting for the completion of the execution of at least one identified operation of the multiple operations of the first thread by at least one identified logic element among the multiple logic elements, executing at least one identified operation of a second thread among the multiple concurrent threads by the at least one identified logic element; and when the execution of the at least one identified operation of the first thread is completed, retrieving at least one runtime context value of the first thread from the context storage device, and executing at least one other operation of the multiple operations of the first thread.

[0012] According to a third aspect, a software program product for executing multiple concurrent threads includes: a non-transitory computer-readable storage medium; first program instructions for executing a first thread of the multiple concurrent threads, each concurrent thread implementing a data flow graph including multiple operations, the multiple operations including at least one identified operation; and second program instructions for storing at least one runtime context value of the data flow graph implemented by the first thread in a context storage device while executing the first thread, wherein the context storage device is configured to store multiple thread contexts, each thread context for one of the multiple concurrent threads, wherein, for a concurrent thread of the multiple concurrent threads, each of the multiple thread contexts includes at least one runtime context value of a data flow graph implemented by the concurrent threads; while waiting for the completion of the execution of at least one identified operation of the plurality of operations of the first thread by at least one identified logic element of the plurality of logic elements, executing at least one identified operation of a second thread of the plurality of concurrent threads by the at least one identified logic element; and when the execution of the at least one identified operation of the first thread is completed: retrieving at least one runtime context value of the first thread from a context storage device; and executing at least one other operation of the plurality of operations of the first thread; wherein the first program instructions and the second program instructions are executed by at least one computerized processor from a non-transitory computer-readable storage medium.

[0013] Referring to the first aspect and the second aspect, in a first feasible implementation of the first aspect and the second aspect, the context storage device includes a plurality of context entries, each context entry being used to store a plurality of runtime context values ​​of a thread context in the plurality of thread contexts. Using the context entry to store the plurality of runtime context values ​​of a thread context in the plurality of thread contexts allows each of the plurality of threads to be resumed independently of the other threads, thereby increasing the flexibility of use of the one or more identified logic elements, and thus reducing the amount of time that the one or more identified logic elements are idle while waiting for the execution of the one or more operations of the first thread to be completed.

[0014] Referring to the first and second aspects, or the first implementation of the first and second aspects, in a second feasible implementation of the first and second aspects, for at least one thread among the multiple concurrent threads, multiple runtime context values ​​of the at least one thread are stored in more than one context entry among the multiple context entries. Using more than one context entry to store multiple runtime context values ​​of a thread allows the thread to have a larger context, which has a certain amount of runtime context values ​​that are not accommodated in a single entry, which improves the availability of the system compared to storing multiple runtime context values ​​of the thread in a single entry. Optionally, the multiple context entries are organized in a table having multiple rows, one row for each context entry in the multiple context entries. Optionally, each of the multiple rows has multiple columns, so that each of the multiple runtime context values ​​of the thread context stored in the row is stored in a column in the multiple columns. Organizing the multiple context entries in a table increases the ease of use of the multiple context entries, thereby allowing the values ​​to be referenced by the index number of the entry and additionally or alternatively by the index number of the column. Optionally, at least one reconfigurable processing grid is further configured to: store at least one runtime context value of the first thread in at least one identified column of the context storage device while executing the first thread; and store at least one other runtime context value of the first thread in at least one identified column of the context storage device. Reusing context entries allows for a reduced amount of storage required to implement the context storage device, thereby reducing implementation costs. Optionally, the data flow graph includes a plurality of nodes and a plurality of edges. Optionally, at least one of the plurality of nodes implements a lookup table, and configuring the at least one reconfigurable processing grid to execute the first thread includes storing the lookup table in at least one other column of the plurality of columns. Storing the lookup table in one or more columns of the context storage device allows for faster access to values ​​of the lookup table than if implemented in application memory, thereby reducing the amount of time a thread takes to access values ​​in the lookup table, and additionally or alternatively reducing the amount of time a context is created for a thread when the context value is driven from the lookup table.

[0015] With reference to the first and second aspects, or the first implementation of the first and second aspects, in a third feasible implementation of the first and second aspects, at least one reconfigurable processing grid is further configured to: when the execution of at least one identified operation of the first thread is completed, store at least one result value in a context entry among the plurality of context entries, wherein the context entry is used to store at least a portion of the thread context of the first thread, and the at least one result value is a result of executing the at least one identified operation of the first thread. Storing the result value in the context entry improves the accuracy of the thread context when resuming after the execution of one or more identified operations is completed.

[0016] With reference to the first and second aspects, or the first implementation of the first and second aspects, in a fourth feasible implementation of the first and second aspects, a first context entry of the plurality of context entries stores a plurality of runtime context values ​​of the first thread, and the at least one reconfigurable processing grid is further configured to: calculate an indication of first context completeness according to the result of at least one test applied to the plurality of runtime context values ​​of the first context entry, and retrieve the at least one runtime context value of the first thread and perform at least one other operation subject to the constraint of the indication of first context completeness. Optionally, the at least one reconfigurable processing grid further comprises: a scheduling circuit for applying at least one test to the plurality of runtime context values. Optionally, applying the at least one test to the plurality of runtime context values ​​comprises: the scheduling circuit executing a test instruction set. Using the scheduling circuit executing the test instruction set allows more than one test to be implemented by executing a different test instruction set for each test, thereby facilitating improving the accuracy of the result of applying one or more tests, and thus improving the accuracy of the indication of first context completeness. Optionally, the first context entry comprises a plurality of valid bits, each valid bit being associated with one of the plurality of runtime context values; and applying the at least one test to the plurality of runtime context values ​​comprises applying the identified bitwise mask to the plurality of valid bits. Applying the bitwise mask to the plurality of valid bits reduces the time required to check the plurality of validity values ​​of the plurality of runtime context values. Optionally, the at least one reconfigurable processing grid is further configured to select the first thread to execute at least one other operation of the plurality of operations of the first thread according to a scheduling policy. The selection of the first thread by the one or more reconfigurable processing grids reduces the delay until the execution of the one or more other operations compared to the selection of the first thread by processing circuitry external to the processing grids, for example, when the one or more hardware processors are not part of the one or more processing grids. Optionally, before selecting the first thread, the at least one reconfigurable processing grid is further configured to calculate another identification of at least one other context completion based on at least one other result of applying at least one test to at least one other plurality of runtime context values ​​of at least one other context entry before selecting the first thread. Identifying the one or more other context completions before selecting the first thread allows for flexible selection of which thread to execute, for example, the context is another thread of the one or more other contexts, which helps improve overall system performance compared to being limited to selecting the first thread. Optionally, at least one reconfigurable processing grid is further configured to: subject to a tag added to one or more context entries of a plurality of context entries, wherein the one or more context entries are used to store at least a portion of a thread context of a first thread, perform at least one of the following: deny execution of at least one other operation of a plurality of operations of the first thread; and provide at least one of the plurality of thread context values ​​of the first thread to at least one other software object.Using tags allows: flexibility in selecting which thread to execute and by whom, which helps improve overall system performance compared to being limited to restarting execution of the first thread and terminating the thread after the first thread is restarted.

[0017] With reference to the first aspect and the second aspect, in a fifth possible implementation of the first aspect and the second aspect, the data flow graph includes a plurality of nodes and a plurality of edges. Optionally, at least one identified operation is represented in the data flow graph by at least one identified node among the plurality of nodes. Optionally, at least one hardware processor is further configured to identify a subgraph (residual subgraph) in the data flow graph, such that the residual subgraph is composed of a node subset of the plurality of nodes and an edge subset of the plurality of edges, wherein, in the data flow graph, there is no path between any two nodes in at least one of the identified nodes, wherein, for each node in the node subset, in the data flow graph, there is no path between at least one identified node and the node, and wherein, for each edge in the edge subset, in the data flow graph, there is no path between at least one identified node and the edge, and at least one runtime context value is at least one edge value of at least one edge in the edge subset. Using a residual subgraph as a context for the flow, wherein there is no path in the data flow graph between one or more identified nodes and any node in the residual subgraph, when the execution of the one or more identified operations is completed, this increases the accuracy of the context because the execution of other parts of the data flow graph that are not the residual subgraph does not affect the execution of the residual subgraph, and vice versa. Optionally, the data flow graph is a directed graph. Optionally, each of the plurality of edges has a head node from the plurality of nodes and a tail node from the plurality of nodes, the node subset includes one or more entry nodes, such that each of the one or more entry nodes is an entry node of the residual subgraph, wherein the entry node is not the head node of any edge in the edge subset, and at least one runtime context value is at least one input value of at least one of the one or more entry nodes.

[0018] Referring to the first aspect and the second aspect, in a sixth feasible implementation of the first aspect and the second aspect, at least one identified operation includes at least one of the following: a memory access operation, a floating-point mathematical operation, execution of another computational graph, access to a coprocessor, and access to a peripheral device connected to at least one reconfigurable processing grid.

[0019] Referring to the first aspect and the second aspect, in a seventh feasible implementation of the first aspect and the second aspect, the plurality of concurrent threads are a subset of a concurrent thread set, and each of the concurrent thread set implements a data flow graph. Optionally, the system further comprises: a buffer storage device for storing a plurality of additional thread contexts, each thread context being used for at least one concurrent thread in the concurrent thread set. Optionally, wherein at least one hardware processor is further configured to further configure at least one reconfigurable processing grid to perform the following operations: storing one or more additional runtime context values ​​of one or more waiting threads in the buffer storage device, wherein the one or more waiting threads are not members of the plurality of concurrent threads; and identifying that the execution of at least one additional thread in the plurality of concurrent threads has been completed in each iteration of the plurality of iterations, retrieving at least one additional runtime context value of at least one waiting thread from the buffer storage device for at least one of the one or more waiting threads, and adding at least one waiting thread to the plurality of concurrent threads for execution by the plurality of logic elements. Using a buffer storage device to store one or more additional runtime context values ​​for a waiting thread that is not a member of the plurality of concurrent threads allows: reusing a plurality of logic elements to execute more threads than the context storage device supports at one time, thereby further improving the overall performance of the system in terms of reducing latency, and additionally or alternatively improving throughput. Optionally, at least one reconfigurable processing grid further includes: a register circuit for tracking the one or more waiting threads; and an additional scheduling circuit for managing the execution of the plurality of concurrent threads. Optionally, the additional scheduling circuit is configured to: select at least one waiting thread from the register circuit; retrieve at least one additional runtime context value for the at least one waiting thread from the buffer storage device; and add the at least one waiting thread to the plurality of concurrent threads for execution by the plurality of logic elements. Using a register circuit to track the one or more waiting threads and using the additional scheduling circuit to manage the execution of the plurality of concurrent threads reduces the latency of scheduling one or more threads of the plurality of concurrent threads for execution compared to managing the execution of one or more hardware processors. Optionally, adding the at least one waiting thread to the plurality of concurrent threads includes: storing the at least one additional runtime context value for the at least one waiting thread in the context storage device. Optionally, the additional scheduling circuit is further configured to: associate each of the at least one waiting thread with a context identification value, the context identification value indicating the thread context of the waiting thread in the context storage device. Optionally, the additional scheduling circuit is further configured to: in a first iteration of the multiple iterations, associate the identified context identification value with a first waiting thread in the one or more waiting threads; in a second iteration of the multiple iterations, identify that the execution of the first waiting thread has been completed; and associate the identified context identification value with a second waiting thread in the one or more waiting threads.Optionally, the buffer storage device includes a plurality of buffer entries, each buffer entry is used to store the thread context of at least one of the one or more waiting threads. Optionally, the registration circuit includes a plurality of registration entries, each registration entry is used to track at least one of the one or more waiting threads. Optionally, the additional scheduling circuit is further configured to: for at least one group of waiting threads in the one or more waiting threads, generate a common thread context associated with each of the at least one group of waiting threads in the buffer storage device; generate a common registration entry associated with each of the at least one group of waiting threads in the registration circuit; and when a new thread in the at least one group of waiting threads is selected from the registration circuit as the at least one waiting thread, calculate at least one private context value of the new thread. Optionally, the registration circuit is further configured to: stop the execution of at least some threads in the concurrent thread set until the number of the one or more waiting threads exceeds a threshold. Waiting for the number of the one or more waiting threads to exceed the threshold allows: adding the one or more waiting threads to the multiple concurrent threads in batches, thereby reducing the overhead of such a configuration, which helps to further improve system performance compared to executing the waiting threads when the waiting threads are available.

[0020] Referring to the first aspect and the second aspect, in an eighth feasible implementation of the first aspect and the second aspect, the plurality of logic elements are a plurality of reconfigurable logic elements, the plurality of reconfigurable logic elements are organized into a plurality of computational groups, and at least one identified logic element is a subset of the plurality of computational groups.

[0021] With reference to the first aspect and the second aspect, in a ninth feasible implementation of the first aspect and the second aspect, the runtime context value of the data flow graph is an input value or an output value of a node among the plurality of nodes of the data flow graph.

[0022] Referring to the first aspect and the second aspect, in a tenth feasible implementation of the first aspect and the second aspect, at least one hardware processor is further configured to configure at least one reconfigurable processing grid to execute the first thread in each of the multiple thread iterations. Optionally, the context storage device includes at least one additional context entry to store additional multiple runtime context values ​​of the data flow graph, wherein the additional multiple runtime context values ​​are common to the multiple thread iterations; and when the execution of at least one identified operation of the first thread is completed, the reconfigurable processing grid is further configured to retrieve at least one of the additional multiple runtime context values ​​from the context storage device. Compared with copying the additional context value for more than one thread iteration in the multiple thread iterations, using one or more additional context entries to store the additional context value that is common to the multiple thread iterations allows the size of the context storage device to be reduced, thereby reducing the implementation cost.

[0023] Referring to the first aspect and the second aspect, in an eleventh feasible implementation of the first aspect and the second aspect, at least one reconfigurable processing grid further comprises at least one other context storage device. Optionally, at least one hardware processor is further configured to: configure at least one reconfigurable processing grid so that when the execution of at least one identified operation of the first thread is completed, at least one additional runtime context value of the first thread is stored in at least one other context storage device, further execute the first thread, and while further executing the first thread, while waiting for at least one other identified logic element in the plurality of logic elements to further execute at least one other identified operation of the plurality of operations of the first thread, at least one other identified operation of another second thread in the plurality of threads is executed by the at least one other identified logic element. Using more than one context storage device allows: cascading the separate scheduling of more than one unpredictable delay operation, thereby further improving system performance in terms of reducing latency, and additionally or alternatively increasing throughput.

[0024] Referring to the first aspect and the second aspect, in a twelfth feasible implementation of the first aspect and the second aspect, at least one reconfigurable processing grid further comprises at least one counter, the plurality of concurrent threads comprises a group of concurrent threads associated with the at least one counter, and at least one runtime context value comprises at least one counter value read from the at least one counter by accessing the at least one counter. Maintaining a common counter associated with a group of concurrent threads allows: providing a unique value for each concurrent thread in the group of concurrent threads, where other context values ​​are common, thereby improving the accuracy of operation of the group of concurrent threads. Optionally, each concurrent thread in the group of concurrent threads implements an identified data flow graph. Optionally, accessing at least one counter comprises an atomic access, the atomic access comprising: reading at least one counter and incrementing at least one counter. Optionally, at least one counter is a sequence of counters. Optionally, incrementing at least one counter comprises: incrementing a first counter in the sequence of counters using a modulo operation, and subject to the constraint that the first counter wraps around after being incremented, incrementing a second counter in the sequence of counters that is consecutive to the first counter. Optionally, at least one counter is a counter sequence, and incrementing at least one counter includes: incrementing a first counter in the counter sequence, and subject to the first counter exceeding a maximum value, incrementing a second counter in the counter sequence that is consecutive to the first counter, and performing at least one of the following: setting the first counter to a new value calculated using the second counter, and setting the maximum value to another new value calculated using the second counter.

[0025] Other systems, methods, features and advantages of the present disclosure will be apparent or become apparent to those of ordinary skill in the art after reviewing the following drawings and detailed description. It is intended that all such additional systems, methods, features and advantages are included in this specification, within the scope of the present disclosure, and protected by the appended claims.

[0026] Unless otherwise defined, all technical terms and / or scientific terms used herein have the same meanings as those generally understood by those of ordinary skill in the art to which the embodiments belong. Although methods and materials similar or equivalent to the methods and materials described herein may be used in the practice or testing of the embodiments, exemplary methods and / or materials are described below. In the event of a conflict, the patent specification, including definitions, shall prevail. In addition, materials, methods, and examples are illustrative only and are not intended to be necessarily restrictive. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Some embodiments are described herein with reference to the accompanying drawings by way of example only. With specific reference now to the drawings, it is emphasized that the details shown are by way of example and are used for illustrative discussion of the embodiments. In this regard, the description in conjunction with the drawings makes it clear to those skilled in the art how the various embodiments can be practiced.

[0028] In the attached picture:

[0029] Figure 1 is a schematic block diagram of an exemplary data flow graph according to some embodiments;

[0030] Figure 2 is a schematic block diagram of an exemplary system according to some embodiments;

[0031] Figure 3 is a schematic block diagram of an exemplary context storage device according to some embodiments;

[0032] Figure 4 is a schematic block diagram of another exemplary system according to some embodiments;

[0033] Figure 5 is a flow chart schematically illustrating an optional operation flow according to some embodiments;

[0034] Figure 6 is a flow chart schematically illustrating an optional operational flow for processing a grid according to some embodiments;

[0035] Figure 7 is a flowchart schematically illustrating another optional operation flow for processing a grid according to some embodiments; and

[0036] Figure 8is a flow chart schematically illustrating yet another optional operation flow for processing a grid according to some embodiments. DETAILED DESCRIPTION

[0037] In the field of computer programming, the term "context" refers to the state that exists when an operation is performed. For a thread, the context of a thread is the set of values ​​that the thread can access when performing an operation. When a thread is executed by a central processing unit (CPU), the context of the thread includes, among other values, multiple values ​​of multiple registers of the CPU, one or more values ​​of thread local storage (TLS), multiple values ​​of stack memory, and a program counter value.

[0038] When a first thread includes an operation with inconsistent latency, the CPU can remain idle while waiting for the operation to complete. In order to improve the performance of a system executing threads, it is usually practiced to configure the CPU to execute a second thread while waiting for the operation of the first thread to complete. There are methods for storing the context of a thread when the thread is suspended by the CPU, and restoring the context of the thread when the thread is resumed by the CPU. When the thread is executed by the CPU, the values ​​accessible to the thread reside in the registers of the CPU or are accessible via the address bus. Therefore, the context of the thread executed by the CPU is determined by the resources available in the CPU, and is generally similar for all threads executed by the CPU. Regardless of the function of the thread, the context of the thread includes the values ​​of an identified register set and an identified stack memory size.

[0039] As the demand for high-performance computer systems providing high throughput and low latency increases, when implementing software applications, large-scale parallel programming paradigms are increasingly used. In such programming paradigms, a larger program is divided into a plurality of smaller problems that can be solved in parallel, and each smaller problem can be solved individually without depending too much on data, and additionally or alternatively not depending too much on the control between a plurality of smaller problems. A software program (program) may include one or more such parallel regions, wherein the execution of the program is divided into a plurality of concurrent threads from a thread executed in sequence (serially), and each concurrent thread performs the task of solving the smaller problem. This is also referred to as a fork operation. When a plurality of concurrent threads are completed, the execution of the program returns to a thread executed serially. This process is referred to as connecting a plurality of concurrent threads to a serial thread.

[0040] While there is no exact threshold amount of concurrent threads that determines when a computation becomes massively parallel, for a given system, there is a certain amount of concurrent threads such that the generation of thread and process contexts and the execution of additionally or alternatively concurrent threads (e.g., where there are more concurrent threads than processing circuits) requires significant processing resources in terms of memory and computation time, thereby increasing the overhead of executing the program.

[0041] One existing solution to reduce context switching overhead is to use a dataflow graph when implementing a distributed method, such as in a distributed implementation of an execution loop. The use of multiple concurrent threads, each of which implements a dataflow graph, allows each of the multiple concurrent threads to be projected onto a portion of a reconfigurable processing grid (processing grid) to be executed simultaneously. In addition, because the multiple concurrent threads implement a common dataflow graph for each thread executed using the thread's data, a projection of the dataflow graph onto the processing grid can be used to execute more than one of the multiple concurrent threads in a pipeline, thereby providing a dataflow graph for each thread input data of the thread. The use of a dataflow graph in the processing grid allows distributed processing to be extended to a greater extent than can be achieved using multiple CPUs, at least in part because of the pipeline nature of executing at least some of the multiple concurrent threads on a projection of the dataflow graph in the processing grid. However, a system may include more threads than can be executed simultaneously on the processing grid.

[0042] Furthermore, the pipeline nature of data flow graph execution is such that when the data flow graph includes an operation with inconsistent latency (inconsistent latency operation), execution of other portions of the data flow graph is halted while waiting for the inconsistent latency operation to complete for threads in the plurality of concurrent threads. These other portions may include at least some of the plurality of concurrent threads executed by projecting the data flow graph onto a processing grid. As a result, execution of one or more of at least some of the threads may be delayed, thereby reducing throughput of the system, while processing resources (in the processing grid) are idle.

[0043] In addition, when a thread implementing a data flow graph is projected to a processing grid, the values ​​accessible to the thread may be located in any logic element of the processing grid and are determined by the projection of the thread. When a data flow graph includes operations with inconsistent delays (inconsistent delay operations), one or more logic elements manipulated to execute a thread cannot be reused to execute another thread implementing another data flow graph without being manipulated again. Typically, the amount of time waiting for an inconsistent delay operation of a first thread to complete is sufficient to execute a second thread, but is insufficient to: reconfigure the processing grid, execute the second thread, and reconfigure the processing grid again to restart execution of the first thread. Therefore, execution of the second thread may be delayed, thereby reducing the throughput of the system, while processing resources (in the processing grid) are idle.

[0044] In at least some embodiments of the present disclosure, the present disclosure solves the technical problem of reducing the amount of time that processing resources in a processing grid are idle, for example, while waiting for the completion of execution of inconsistent delay operations, in particular, when multiple concurrent threads are executed in a pipeline, and the multiple concurrent threads implement a data flow graph including one or more identified operations. By reducing the amount of time that processing resources in a processing grid are idle, at least some embodiments described herein improve the performance of the system, for example, reduce the latency of the system compared to a standard method of executing multiple concurrent threads on a processing grid, and additionally or alternatively increase the throughput of the system.

[0045] To this end, the present disclosure proposes that, in some embodiments of the present disclosure, a thread context storage device for each concurrent thread among multiple concurrent threads is stored in a context storage device, and the context storage device is used to suspend and resume the execution of one or more concurrent threads among the multiple concurrent threads without manipulating the processing grid.

[0046] Unless otherwise specified, for the sake of brevity, the term "context" is used to mean "thread context", and the two terms can be used interchangeably. Optionally, the thread context includes a set of values, each input value is an input value input into a node of a data flow graph or an output value output to a node of a data flow graph. In such an embodiment, the present disclosure proposes to store one or more runtime context values ​​of a first thread in a context storage device, and the first thread executes a data flow graph including one or more identified operations. The runtime context value of a thread is a value accessible to the thread while the thread is executing. Optionally, at least one of the one or more identified operations is an inconsistent delay operation. Optionally, while waiting for one or more identified logic elements of a plurality of logic elements of a processing grid to complete the execution of one or more identified operations of the first thread, the present disclosure proposes that one or more identified operations of a second thread be executed by one or more identified logic elements. Optionally, the second thread is executed concurrently with the first thread in the pipeline by one or more identified logic elements. Thus, rather than the one or more identified logic elements remaining idle while waiting for the one or more identified operations of the first thread to complete, the one or more identified logic elements may be used to execute the one or more identified operations of the second thread, and additionally or alternatively, while the one or more identified logic elements remain idle because the entire pipeline is stalled.

[0047] There is no need to reconfigure the processing grid, that is, there is no need to manipulate one or more identified logic elements because they are already configured to perform one or more identified operations. Optionally, when the execution of one or more identified operations of the first thread is completed, the present disclosure proposes to retrieve one or more runtime context values ​​of the first thread from the context storage device and restart the execution of the first thread, that is, to execute one or more other operations of the first thread. Optionally, before executing one or more other operations, one or more runtime context values ​​are loaded into one or more logic elements of the processing grid. One or more runtime context values ​​of the first thread are stored in the context storage device before executing one or more identified operations of the second thread, and one or more runtime context values ​​are retrieved when the execution of one or more identified operations of the first thread is completed, which allows the context of the first thread to be retained, so that the execution of the first thread is restarted after the execution of one or more identified operations is completed without being affected by the execution of one or more identified operations of the second thread using one or more identified logic elements. This provides the following benefits: reducing the amount of time that one or more identified logic elements are idle and waiting for the execution of one or more operations of the first thread to be completed, thereby promoting an increase in system throughput and reducing system delays when executing one or more tasks of the system.

[0048] Optionally, one or more identified operations are represented in the data flow graph by one or more identified nodes in the multiple nodes of the data flow graph. The context of one or more identified operations includes a subgraph (residual subgraph) of the data flow graph, which does not contain a path to or from one or more identified nodes. Therefore, the residual subgraph includes a node subset of multiple nodes of the data flow graph, so that there is no path from one or more identified nodes to any node in the node subset in the data flow graph, and vice versa. In addition, the residual subgraph includes an edge subset of multiple edges of the data flow graph, so that there is no path from one or more identified nodes to any edge in the edge subset in the data flow graph, and vice versa. Use the residual subgraph in which there is no path to or from one or more identified nodes in the data flow graph to define one or more context values, the one or more context values ​​do not affect the execution of one or more identified operations, and are not affected by the results of executing one or more identified operations, so that the values ​​required for the execution of the restarted thread are allowed to be correctly restored after the execution of one or more identified operations is completed. Furthermore, unlike a thread executing on a CPU, whose context includes values ​​associated with predefined named general registers of the CPU, the thread context of a thread implementing a dataflow graph and executing on a processing grid includes values ​​derived from the structure of the dataflow graph rather than from the structure of the circuitry executing the thread.

[0049] Optionally, the one or more identified operations include more than one operation having inconsistent delays.

[0050] Optionally, the context storage device includes a plurality of context entries, each context entry in the plurality of context entries being used to store a plurality of runtime context values ​​for a thread context of a concurrent thread in the plurality of concurrent threads. The thread context may include more values ​​than fit in one context entry. Optionally, for at least one thread in the plurality of concurrent threads, the plurality of runtime context values ​​for the thread context of the at least one thread are stored in more than one context entry in the plurality of context entries. Optionally, the plurality of context entries are organized in a table having a plurality of rows, wherein a row is for each context entry in the plurality of context entries. Optionally, the context storage device is a reservation station.

[0051] Optionally, the context storage device is additionally or alternatively used to store a lookup table implemented by a node of a data flow graph implemented by multiple concurrent threads.

[0052] Optionally, the system includes more than one context storage device, thereby allowing a series of identified operations to be individually scheduled. A first context storage device can be used to manage the reuse of a first identified set of logic elements for one or more first identified operations of a thread, and a second context storage device can be used to manage the reuse of a second identified set of logic elements that implement one or more second identified operations of a thread. This provides the following benefits: additional flexibility in scheduling the execution of multiple concurrent threads, thereby further reducing the amount of time that processing resources of the processing grid are idle compared to scheduling the execution of multiple concurrent threads while considering the one or more first identified operations and the one or more second identified operations.

[0053] In addition, in some embodiments of the present disclosure, the present disclosure solves another technical problem of managing the execution of a large number of concurrent threads, which exceeds the amount of threads that a processing grid can execute simultaneously in a given configuration of the processing grid. In such an embodiment, the plurality of concurrent threads is a subset of a set of concurrent threads, wherein each concurrent thread in the set of concurrent threads implements a data flow graph. Compared to standard methods of scheduling a large number of concurrent threads, at least some embodiments described herein improve the performance of the system by reducing the amount of context switching and scheduling overhead.

[0054] To this end, in some embodiments described in the present disclosure, the present disclosure proposes storing multiple additional thread contexts in a buffer storage device, wherein each thread context is for one concurrent thread in a concurrent thread set, and in each iteration of multiple iterations, retrieving additional thread contexts of additional threads in the concurrent thread set from the buffer storage device, and using the additional thread contexts when adding other threads to the multiple threads for execution by the processing grid.

[0055] Optionally, the set of concurrent threads includes one or more waiting threads, the one or more waiting threads are not members of the plurality of concurrent threads, and the other threads are waiting threads in the one or more waiting threads. Optionally, the waiting thread is added to the plurality of concurrent threads, and its execution is managed using a context storage device, and the context storage device optionally stores other thread contexts retrieved from the buffer storage device. Optionally, the waiting thread is projected to the processing grid without being added to the plurality of concurrent threads, and one or more values ​​of the other thread contexts are optionally loaded into one or more other logic elements of the processing grid. Optionally, the waiting thread is selected in response to identifying that execution of at least one additional thread in the plurality of concurrent threads has completed.

[0056] Optionally, after storing the contexts of the other multiple threads in the buffer storage device, multiple concurrent threads are selected from the concurrent thread set.

[0057] Optionally, the processing grid includes circuitry (registration circuitry) for tracking one or more waiting threads. Optionally, the registration circuitry includes a plurality of registration entries, each registration entry being used to track at least one of the one or more waiting threads. In some embodiments described herein, the concurrent thread set includes one or more waiting thread groups, wherein the waiting thread groups have a shared context. For such a waiting thread group, the buffer storage device may have a common thread context associated with each waiting thread in the waiting thread group, and the registration circuitry may have a common registration entry associated with each waiting thread in the waiting thread group. When selecting other threads from the waiting thread group, optionally, when adding the other threads to the plurality of threads for execution by the processing grid, one or more private context values ​​of the other threads are generated and used.

[0058] To this end, in some embodiments described in the present disclosure, the present disclosure proposes storing multiple additional thread contexts in a buffer storage device, wherein each thread context is for one of the concurrent thread sets, and in each iteration of multiple iterations, retrieving additional thread contexts of additional threads in the concurrent thread set from the buffer storage device, and using the additional thread contexts when adding the other threads to the multiple threads for execution by the processing grid.

[0059] Optionally, the system includes more than one buffer storage device, which allows for separate scheduling of more than one concurrent thread set, thereby implementing more than one data flow graph. A first buffer storage device can be used to manage the scheduling of a first concurrent thread set, and a second buffer storage device can be used to schedule a second concurrent thread set. This provides the following benefits: providing additional flexibility in scheduling the execution of more than one concurrent thread set independently of each other, further reducing the amount of time that processing resources of the processing grid are idle compared to scheduling the execution of one or more concurrent thread sets as a single thread set.

[0060] It should be noted that some embodiments according to the present disclosure solve the above two technical problems, while some other embodiments solve one or the other of the problems.

[0061] Before describing at least one embodiment in detail, it should be understood that the embodiments are not necessarily limited to the details of the construction and arrangement of the components and / or methods described in the following description and / or shown in the drawings and / or examples in the application of the present disclosure. The implementation described in the present disclosure can be other embodiments or can be practiced or executed in various ways.

[0062] The embodiments may be systems, methods, and / or computer program products. The computer program product may include: a computer-readable storage medium (or medium) having computer-readable program instructions to cause a processor to perform aspects of the embodiments.

[0063] A computer-readable storage medium may be: a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the above. The computer-readable storage medium used herein should not be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0064] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0065] The computer-readable program instructions for performing the operation of the embodiment can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code, native compilation or just-in-time compilation (JIT), written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, Java, object-oriented Fortran, etc., interpreted programming languages-such as JavaScript, Python, etc., and conventional process programming languages-such as "C" programming language, Fortran or similar programming languages. The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), coarse-grained reconfigurable architectures (CGRAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits in order to perform aspects of the embodiments.

[0066] The present invention relates to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment. It will be understood that each block in the flowchart and / or block diagram and the combination of blocks in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0067] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a means for implementing the functions / actions specified in the flowchart and / or block diagram blocks. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can instruct the computer, programmable data processing device, and / or other device to operate in a specific manner, so that the computer-readable storage medium storing the instructions includes the following manufactured product, which includes instructions for implementing aspects of the functions / actions specified in the flowchart and / or block diagram blocks.

[0068] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0069] Flowcharts and block diagrams in the accompanying drawings show possible architectures, functions and operations of implementations of systems, methods and computer program products according to various embodiments. In this respect, each block in a flow chart or a block diagram may represent a part of a module, a segment or an instruction, which includes one or more executable instructions for realizing a specified logical function. In some alternative embodiments, the function indicated in the block may not occur in the order indicated in the figure. For example, two blocks of continuous display may actually be performed concurrently in fact, or may sometimes be performed in reverse order, depending on the function involved. It should also be noted that each block in a block diagram and / or a flow chart and the combination of the blocks in a block diagram and / or a flow chart can be realized by a dedicated hardware-based system that performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0070] Reference now Figure 1, a schematic block diagram of an exemplary data flow graph 100 according to some embodiments is shown. Optionally, the data flow graph 100 includes multiple operations. Optionally, the data flow graph 100 includes: multiple nodes, for example, including node 110A1, node 110A2, node 110A3, node 110A4, node 110A5, node 110A6, node 110A7, node 110A8, node 110B1, node 110B2, node 110B3, node 110B4, and one or more identified nodes 110C, collectively referred to as multiple nodes 110. Optionally, the data flow graph 100 includes a plurality of edges, such as edge 120A1, edge 120A2, edge 120A3, edge 120A4, edge 120A5, edge 120A6, edge 120A7, edge 120A8, edge 120A9, edge 120B1, edge 120B2, edge 120B3, edge 120B4, edge 120C1, and edge 120C2, collectively referred to as a plurality of edges 120. Optionally, each of the plurality of edges 120 is a directed edge connecting two of the plurality of nodes 110 and having a head node and a tail node. For example, node 110A2 is the tail node of edge 120B1, and node 110B1 is the head node of edge 120B1.

[0071] Optionally, the plurality of operations include one or more identified operations. Optionally, one or more nodes 110C implement one or more identified operations. Optionally, one or more identified operations have inconsistent delays. Some examples of operations with inconsistent delays include, but are not limited to, memory access operations, floating point math operations, executing another computational graph, access to a coprocessor, and access to a peripheral device connected to a processing grid. Some examples of coprocessors include tensor core units and graphics processing units.

[0072] The context of the data flow graph 100 for one or more identified operations includes a value of a residual subgraph, which optionally includes a node subset of a plurality of nodes 110, wherein no path exists between any two of the one or more identified nodes 110C in the data flow graph 100, and for each node in the node subset, no path exists between the node and the one or more identified nodes 110C in the data flow graph 100. Optionally, the residual subgraph includes an edge subset of a plurality of edges 120, wherein for each edge in the edge subset, no path exists between the edge and the one or more identified nodes 110C in the data flow graph 100. In this example, the residual subgraph 111 includes: a node subset, including node 110B1, node 110B2, node 110B3, and node 110B4, collectively referred to as node subset 110B. In this example, in the data flow graph 100, there is no path between one or more identified nodes 110C and any of the nodes 110B1, 110B2, 110B3, and 110B4. In addition, in this example, the residual subgraph 111 includes: the following edge subsets, including edge 120C1 and edge 120C2, are collectively referred to as edge subsets 120C. In this example, in the data flow graph 100, there is no path between one or more identified nodes 110C and any of the edges 120C1 and 120C2.

[0073] Optionally, the runtime context value of the data flow graph 100 is an input value of a node in the plurality of nodes 120, such as an input value of node 110A2 or node 110B3. Optionally, the runtime context value of the data flow graph 100 is an input value of a node in the residual graph 111, such as an input value of node 110B2 or a value of edge 120B2.

[0074] Optionally, the runtime context value of the data flow graph 100 is an output value of a node in the plurality of nodes 120, such as an output value of node 110B2 or a value of edge 120C1. Optionally, the runtime context value of the data flow graph 100 is an output value of a node in the residual graph 111, such as an output value of node 110B3 or a value of edge 120B4. Optionally, the runtime context value of the data flow graph 100 is an output value of one or more identified nodes 110C.

[0075] In an embodiment, the context of the data flow graph 100 with respect to one or more identified operations includes the value of edge 120B1, the value of edge 120B2, and the value of edge 120A7. In another embodiment, the context of the data flow graph 100 with respect to one or more identified operations includes the value of edge 120B3, the value of edge 120B4, and the value of edge 120A7. In another embodiment, the context of the data flow graph 100 with respect to one or more identified operations includes the value of edge 120B3, the value of edge 120C1, the value of edge 120C2, and the value of edge 120A7.

[0076] An entry node into the graph is a node that is not the head node of any edge of the graph. Optionally, the context of the data flow graph 100 with respect to one or more identified operations only includes the entry node into the residual subgraph and the output values ​​of one or more identified nodes. In this example, such context includes the value of edge 120B1 (or the input value of node 110B1), the value of edge 120B2 (or the input value of node 110B2), and the value of edge 120A7.

[0077] In some embodiments, the system executes multiple concurrent threads, each thread implementing dataflow graph 100 .

[0078] Now also refer to Figure 2 , shows a schematic block diagram of an exemplary system 200 according to some embodiments. In such an embodiment, at least one hardware processor 201 is connected to at least one reconfigurable processing grid 210.

[0079] For the sake of brevity, the term "processing unit" is used hereafter to mean "at least one hardware processor", and these terms are used interchangeably. A processing unit can be: any type of programmable or non-programmable circuit that is configured to perform the operations described below. A processing unit can include hardware and software. For example, a processing unit can include one or more processors and a temporary or non-temporary memory that carries a program that causes the processing unit to perform corresponding operations when the one or more processors execute the program.

[0080] In addition, for simplicity, the term "processing grid" is used hereafter to mean "at least one reconfigurable processing grid", and these terms are used interchangeably. Optionally, the interconnected processing grid includes a plurality of reconfigurable logic elements connected by a plurality of configurable data routing nodes. An example of a reconfigurable logic element is a processing circuit that can be manipulated to perform one of the identified mathematical operations and logical operations. Some examples of mathematical operations are addition and subtraction. Some examples of logical operations are logical negation and logical AND operations. Optionally, processing grid 210 includes: a plurality of logic elements, including logic element 211A, logic element 211B, logic element 211C, logic element 211D, and logic element 211E, collectively referred to as a plurality of logic elements 211. Optionally, the plurality of logic elements 211 are a plurality of reconfigurable logic elements. Optionally, the plurality of logic elements 211 are organized into a plurality of computational groups.

[0081] Optionally, processing grid 210 includes one or more context storage devices, such as context storage device 212, and additionally or alternatively other context storage devices 222. Optionally, context storage device 212 is configured to store multiple thread contexts, each thread context for one of multiple concurrent threads. Optionally, for concurrent threads of thread contexts, each thread context in the multiple thread contexts includes one or more runtime context values ​​of a data flow graph implemented by the concurrent thread. For example, each thread context may include the value of edge 120B1 of the thread of thread context. Optionally, context storage device 212 is a reservation station. Optionally, other context storage devices 222 are reservation stations.

[0082] Now also refer to Figure 3 , shows a schematic block diagram of an exemplary context storage device 212 according to some embodiments. In such an embodiment, the context storage device includes: a plurality of context entries, including, for example, context entry 301A, context entry 301B, context entry 301C, and context entry 301D, collectively referred to as a plurality of context entries 301. Optionally, each of the plurality of context entries 301 is used to store a plurality of runtime context values ​​of a thread context in a plurality of thread contexts. For example, context entry 301D may be used to store a first thread context of a first thread in a plurality of concurrent threads. Optionally, the plurality of runtime context values ​​of the first thread include runtime context value 310A, runtime context value 310B, runtime context value 310C, and runtime context value 310D, collectively referred to as a plurality of runtime context values ​​310. Optionally, context entry 301C is used to store a second thread context of a second thread in a plurality of concurrent threads. Optionally, a plurality of runtime context values ​​of at least one thread context of at least one thread are stored in more than one context entry in the plurality of context entries 301.

[0083] Optionally, the plurality of context entries 301 are organized in a table. Optionally, the table has a plurality of rows. Optionally, each of the plurality of context entries 301 has a row in the plurality of rows. Optionally, each of the plurality of rows has a plurality of columns. Optionally, each of the plurality of runtime context values ​​310 is stored in one of the plurality of columns of the row storing the context entry 301D.

[0084] Now refer to it again Figure 2. Optionally, context storage 212 is implemented in a memory area of ​​processing grid 210. Optionally, context storage 212 is implemented in one or more memory components 230 connected to processing grid 210. Optionally, a portion of context storage 212 is implemented in a memory area of ​​processing grid 210, while another portion of context storage 212 is implemented in one or more memory components 230. Similarly, other context storage 222 may be fully or partially implemented in another memory area of ​​processing grid 210 and one or more of one or more memory components 230.

[0085] Optionally, the processing grid 210 includes: a scheduling circuit 213, optionally connected to the context storage device 212. Optionally, the scheduling circuit 213 is used to calculate an indication that a context in the plurality of contexts is complete. A context of a thread is complete when processing of the thread can be continued from a point where processing was suspended, such as waiting for an inconsistency delay operation to complete. Optionally, the scheduling circuit 213 is used to configure one or more logic elements of the plurality of logic elements 211 to execute one or more threads in the plurality of concurrent threads, optionally by loading one or more runtime context values ​​for the one or more threads retrieved from the context storage device 212 into the one or more logic elements. Optionally, the scheduling circuit 213 is implemented in hardware. Optionally, the scheduling circuit 213 executes a set of scheduling computer instructions. Optionally, the set of scheduling computer instructions includes a set of test instructions.

[0086] Similarly, the processing grid 210 optionally includes a further scheduling circuit 223 , optionally connected to the further context storage 222 , optionally for managing the execution of multiple concurrent threads using the further context storage 222 .

[0087] Optionally, the processing grid 210 includes one or more counters 240. Optionally, the plurality of concurrent threads includes a group of concurrent threads associated with the one or more counters 240. There are implementations in which a thread in the plurality of concurrent threads can identify a task to be performed based on one or more counter values ​​of the one or more counters 240. For example, when the plurality of concurrent threads implements a loop, one of the one or more counters 240 is used to count iterations of the loop. In another example, the plurality of concurrent threads implements two or more nested loops, wherein each of the one or more counters counts iterations of one of the two or more nested loops. The one or more counter values ​​of the one or more counters 240 can indicate which iteration of the loop is to be executed. Optionally, accessing the one or more counters 240 includes an atomic access, which includes: reading the one or more counters 240 and incrementing or decrementing at least one of the one or more counters 240. Therefore, after the first thread accesses the one or more counters 240, at least one of the one or more counters 240 is incremented or decremented, so that each of the plurality of concurrent threads accessing the one or more counters 240 retrieves a unique combination of the one or more counter values.

[0088] Optionally, the one or more counters 240 are a sequence of counters, for example when implementing two or more nested loops. In such an implementation, a first counter of the one or more counters implements a counter of an inner loop of the two or more nested loops, and a second counter of the one or more counters 240 - consecutive to the first counter in the sequence of counters - implements another counter of an outer loop of the two or more nested loops. It should be noted that the designations of "first counter" and "second counter" do not refer to absolute ordinal identifiers in the sequence of counters, but to two consecutive counters in the sequence of counters.

[0089] Optionally, at least one of the one or more nested loops has a target iteration amount. For example, when the one or more counters 240 represent multiple dimensions of a group or multiple tasks, such as an array of tasks. Optionally, incrementing the one or more counters 240 includes: incrementing a first count using a modulus operation. Optionally, each of the one or more counters 240 has a modulus. Not all of the one or more counters 240 share a common modulus. Optionally, incrementing the one or more counters 240 includes: subject to the constraint that the first counter wraps around after being incremented, incrementing a second counter. Optionally, incrementing the one or more counters 240 includes: incrementing a first counter in a sequence of counters, and subject to the constraint that the first counter exceeds a maximum value, incrementing the second counter and setting the first counter to a new value calculated using the second counter. Additionally or alternatively, incrementing the one or more counters 240 includes: setting the maximum value to another new value calculated using the second counter, such as when the inner loop starts according to a value derived from a counter of the outer loop, such as when each iteration of the outer loop includes executing the inner loop a certain number of times, the number of times being derived using the current value of the second counter.

[0090] It should be noted that while the above description focuses on incrementing a counter, similarly the first counter may be decremented and, subject to the first counter being below a minimum value, the second counter may be modified. Optionally, a step value other than 1 is used to increment or decrement the counter.

[0091] In some embodiments, the plurality of concurrent threads is a subset of a set of concurrent threads, each concurrent thread implementing dataflow graph 100. Optionally, the set of concurrent threads includes one or more waiting threads that are not members of the plurality of concurrent threads and are waiting to execute.

[0092] Now also refer to Figure 4 , shows a schematic block diagram of another exemplary system 400 according to some embodiments. In such an embodiment, the processing grid 210 includes a buffer storage device 410. Optionally, the buffer storage device 410 stores a plurality of additional thread contexts, each thread context for one or more thread contexts in the concurrent thread set. Optionally, the buffer storage device 410 is implemented in another memory area of ​​the processing grid 210. Optionally, the buffer storage device 410 is implemented in one or more memory components 230. Optionally, the buffer storage device 410 includes a plurality of buffer entries, wherein each of the plurality of buffer entries is used to store a thread context of at least one waiting thread of the one or more waiting threads in the concurrent thread set.

[0093] Optionally, the processing grid 210 includes a register circuit 411 for tracking one or more waiting threads. Optionally, the register circuit 411 includes a plurality of register entries, each register entry for tracking at least one of the one or more waiting threads. Optionally, the processing grid 210 includes: a buffer scheduling circuit 412 for managing the execution of multiple concurrent threads. Optionally, the buffer scheduling circuit 412 is connected to the buffer storage device 410. Optionally, the buffer scheduling circuit 412 is connected to the register circuit 411. Optionally, the buffer scheduling circuit 412 is connected to the context storage device 212, optionally through one or more inlet ports (not shown).

[0094] Optionally, processing grid 210 includes one or more buffer stores. Optionally, one or more other buffer stores store a plurality of thread contexts, each thread context for one or more of one or more other concurrent thread sets. Similarly, processing grid 210 optionally includes: one or more other buffer scheduling circuits, optionally connected to the one or more other buffer stores, and additionally or alternatively includes one or more other registration circuits, optionally connected to the one or more other buffer scheduling circuits.

[0095] In some embodiments, processing unit 201 configures processing grid 210 to execute multiple concurrent threads, and additionally or alternatively to execute sets of concurrent threads.

[0096] Now also refer to Figure 5 , shows a flow chart schematically representing an optional operational flow 500 according to some embodiments. In such an embodiment, at 510, the processing unit 201 identifies the residual subgraph 111. Optionally, the processing unit 201 uses the residual subgraph 111 to determine a plurality of runtime context values ​​for each of the plurality of concurrent threads, as described above. Figure 1 As described in.

[0097] Optionally, in 520, the processing unit 201 configures the processing grid 210. Optionally, configuring the processing grid 210 includes: manipulating one or more of the plurality of logic elements 211. Optionally, configuring the processing grid 210 includes, but is not limited to: manipulating one or more of the context storage 212, the scheduling circuit 213, the buffer storage 410, the register 411, and the buffer scheduling circuit 412. Optionally, configuring the processing grid 210 includes: loading one or more values ​​into one or more of the context storage 212 and the buffer storage 410.

[0098] In some embodiments, processing unit 201 configures processing grid 210 to implement one or more methods. Figure 6, a flowchart schematically representing an optional operational flow 600 for a processing grid according to some embodiments is shown. Optionally, processing unit 201 configures processing grid 210 to implement method 600. In such an embodiment, in 601, processing grid 210 uses at least some of multiple logic elements 211 to execute a first thread of multiple concurrent threads. Optionally, one or more identified operations of data flow graph 100 are performed by one or more identified logic elements of multiple logic elements 211, such as identified logic element 211C. Optionally, when multiple logic elements 211 are organized in multiple computation groups, one or more identified logic elements are subsets of the multiple computation groups. In 610, processing grid 201 optionally stores one or more runtime context values ​​of the first thread, such as including runtime context value 310A, in context storage 212, such as in context entry 301D. When the plurality of context entries 310 are organized in a table, wherein the context entry 301D is a row in the table, the processing grid 201 optionally stores the runtime context value 310A in one of the columns of the row. Optionally, the runtime context value 310A is an input value of an entry node into the residual subgraph 111, such as the input value of the node 110B1. Optionally, when the context entry 301D is a row in the table, the processing grid 210 stores the runtime context value 310A in one or more identified columns of the context storage 212. Optionally, the processing grid 201 stores the runtime context value 310A while waiting for the execution of the one or more identified operations of the first thread by the identified logic element 211C to complete. Additionally, while waiting for the one or more identified operations of the first thread to be executed by the identified logic element 211C to complete, in 612, the processing grid 210 optionally configures at least some logic elements to execute a second thread of the plurality of concurrent threads, such that the processing grid 210 executes the one or more identified operations of the second thread to be executed by the identified logic element 211C.

[0099] Optionally, when the execution of the one or more identified operations of the first thread is completed, the processing grid 210 optionally stores one or more result values ​​in the context entry 301D in 615. Optionally, the one or more result values ​​are the result of executing the one or more identified operations. For example, when the one or more identified operations include memory accesses, the one or more result values ​​may include data values ​​retrieved by the memory accesses.

[0100] In 620, the processing grid 210 optionally calculates an indication that the context entry 301D is complete. Optionally, the indication that the context entry 301D is complete is calculated based on the result of applying one or more tests to the plurality of runtime context values ​​310. Optionally, the one or more tests are applied by the scheduling circuit 213. Optionally, the scheduling circuit executes a test instruction set to apply the one or more tests. Optionally, applying the one or more tests includes: checking whether the plurality of runtime context values ​​310 are valid. According to some embodiments, to achieve this, the context entry 301D includes a plurality of valid bits. Optionally, each of the plurality of valid bits is associated with one of the plurality of runtime context values ​​310. Optionally, applying the one or more tests includes: applying an identified bitwise mask to the plurality of valid bits, wherein the identified bitwise mask indicates which of the plurality of runtime context values ​​310 needs to be valid to make the context entry 301D complete. In some embodiments, the identified bitwise mask is applied to the plurality of valid bits describing a row of the context entry table.

[0101] Optionally, processing grid 210 selects the first thread for execution at 630. Optionally, processing grid 210 uses scheduling circuitry 213 to select the first thread.

[0102] More than one thread in the plurality of concurrent threads may complete executing the one or more identified operations of each of the more than one threads. Optionally, the first thread is selected according to a scheduling policy. For example, the first thread may be the first thread found. Optionally, the scheduling policy includes using a loop to prevent thread starvation. Optionally, the scheduling policy is based on a timestamp, such as scheduling the earliest or latest completed thread. Optionally, the scheduling policy is based on a priority value. Optionally, a priority value is assigned to the thread. Optionally, the priority value is calculated by the thread for itself. Optionally, the priority value determines a partial priority order among the plurality of concurrent threads.

[0103] Optionally, in 622, processing grid 210 identifies that more than one thread has completed executing the one or more identified operations of each of the more than one threads by calculating another indication that one or more other context entries in plurality of context entries 301 have completed. Optionally, the other indication is calculated based on one or more other results of applying one or more tests to one or more other runtime context values ​​of one or more other context entries. Optionally, processing grid performs 622 before selecting the first thread in 630, for example when a priority-based scheduling policy is used.

[0104] In 632, the processing grid 210 optionally retrieves the runtime context value 310A from the context storage 212. Optionally, subject to the completion of the context entry 301D, the processing grid 210 performs 632. Optionally, the processing grid 210 retrieves the runtime context value 310A from the context storage 212 using the scheduling circuit 212. Optionally, in 640, the processing grid 210 resumes execution of the first thread by performing one or more other operations of the plurality of operations of the first thread, such as one or more operations implemented in the node 110A6. Optionally, the processing grid loads the retrieved runtime context value 310A to one or more of the plurality of logic elements 211. Optionally, the processing grid 210 performs 640 using the scheduling circuit 212. Optionally, subject to the completion of the context entry 301D, the processing grid 210 performs 640.

[0105] In some embodiments, it may be necessary to suspend the execution of the first thread. Some examples include breakpoints set for debugging. Optionally, the breakpoints are conditional based on one or more values ​​of multiple runtime context values ​​311. When suspending the execution of the first thread, the processing grid 210 can stop (halt) the execution of the first thread. Additionally or alternatively, the processing grid 210 can provide context information about the first thread to one or more other software objects executed in the system 200, some examples of which are debuggers and loggers. Optionally, subject to the constraints of the tags added to the context entry 301D, the processing grid 210 refuses to execute one or more other operations of the first thread. Additionally or alternatively, subject to the constraints of the tags, the processing grid 210 optionally provides one or more of the multiple runtime context values ​​310 to one or more other software objects. Optionally, the one or more other software objects are implemented in another part of the processing grid 210. Optionally, the one or more other software objects are executed by the processing unit 201. Optionally, the one or more other software objects are executed by another processing unit (not shown) connected to the processing grid 210.

[0106] In some embodiments, context entries may be reused. For example, when multiple context entries 301 are organized in a table, one of the multiple columns of a row in the table may be reused. In such an embodiment in 650, the processing grid 210 optionally stores one or more other runtime context values ​​of the first thread in a column for storing one or more runtime context values ​​including the runtime context value 310A in 610. The context entries may be reused when executing another identified operation of the first thread with inconsistent latency. Optionally, the context entries are reused while waiting for another execution of one or more identified operations of the first thread to complete. Optionally, the first thread is executed in each iteration of multiple iterations. Optionally, the processing grid 210 executes 610 in one iteration of the multiple iterations and executes 650 in another iteration of the multiple iterations.

[0107] In some embodiments, one or more nodes of data flow graph 100 implement a lookup table. When multiple context entries 301 are organized in a table, configuring processing grid 210 to execute the first thread optionally includes storing the lookup table in one or more other columns of the multiple columns of the table, rather than the one or more columns used to store runtime context values ​​310A. When the lookup table is stored in the other one or more columns, the other one or more columns can be read by any of the multiple concurrent threads, but cannot be written.

[0108] Optionally, the thread accesses the lookup table using an instruction including an index and a value of the other column or columns. Optionally, when the index identifies a column that is part of the lookup table, the value is used as an indication of a row to access.

[0109] Optionally, the processing grid 210 is also configured by the processing unit 201 to execute the first thread in each of the multiple thread iterations. In some embodiments, one or more additional context entries in the multiple context entries 301 are used to provide at least a portion of the context of the first thread, which is common to each of the multiple thread iterations of executing the first thread. Optionally, the one or more additional context entries include one or more additional context values, which are entry values ​​in the residual subgraph, such as the input value of node 110B2. Optionally, one or more other context entries provide another portion of the context for executing the thread for multiple thread iterations, including one or more other context values, which are located inside the residual subgraph, such as the value of edge 120C2, and may be different for some of the multiple thread iterations. Optionally, the processing grid 210 is configured to: when the execution of the one or more identified operations of the first thread is completed, retrieve at least one of the one or more additional context values ​​from the context storage device 212, optionally for the next thread iteration for executing the first thread. Optionally, at least one of the one or more additional context values ​​is loaded into one or more of the plurality of logic elements 211 before other values ​​of the one or more additional context values ​​are loaded.

[0110] In some embodiments including more than one context store, processing unit 201 configures processing grid 210 to perform the following optional method.

[0111] Now also refer to Figure 7 , a flow chart schematically representing another optional operation flow 700 for a processing grid according to some embodiments is shown. In such an embodiment, when the execution of one or more identified operations of a first thread is completed, in 710, the processing grid 210 stores one or more and additional runtime context values ​​of the first thread in the other context storage device 222. In 720, the processing grid 210 optionally further executes the first thread, and while further executing the first thread in 730, the processing grid 210 optionally executes one or more other identified operations of another second thread through one or more other identified logic elements, such as logic element 211E, while waiting for the completion of the execution of the one or more other identified operations of the first thread by the logic element 211E. Optionally, the processing grid 210 manages the execution of multiple concurrent threads by using the other context storage device 222 to use the other scheduling circuit 223.

[0112] As described above, in some embodiments described herein, the plurality of concurrent threads is a subset of a set of concurrent threads, each of which implements the data flow graph 100. In such embodiments, the set of concurrent threads includes one or more waiting threads that are not members of the plurality of concurrent threads and are waiting to be executed. Optionally, the processing grid 210 uses the buffer storage device 410 to manage the execution of the set of concurrent threads. Optionally, in order to manage the execution of the set of concurrent threads, the processing unit 201 configures the processing grid 210 to perform the following method.

[0113] Now also refer to Figure 8 , a flowchart schematically representing another optional operational flow 800 for a processing grid according to some embodiments is shown. Optionally, in 810, the processing circuit 210 stores one or more additional runtime context values ​​for one or more waiting threads in the buffer storage device 410. The following practice may be beneficial: starting execution of the concurrent thread set only when there are enough waiting threads to increase utilization of multiple logic elements while waiting for the completion of execution of one or more identified operations of one or more concurrent threads in the concurrent thread set. Therefore, in 805, the processing grid 210 optionally stops execution of at least some of the concurrent threads in the concurrent thread set until the amount of one or more waiting threads exceeds a threshold. Optionally, the processing grid 210 uses the registration circuit 411 to stop execution of at least some of the concurrent threads in the concurrent thread set.

[0114] Optionally, in each of the plurality of iterations, processing grid 210 identifies that execution of one or more additional threads in the plurality of concurrent threads has completed in 815. Optionally, processing grid 210 selects at least one waiting thread from registration circuit 411 in 820, and processing grid 210 optionally retrieves one or more additional runtime context values ​​for the at least one waiting thread from buffer storage 410 in 822. Optionally, processing grid 210 uses buffered scheduling circuit 412 to retrieve the one or more additional runtime context values.

[0115] Optionally, processing grid 210 adds the completed one or more additional threads to one or more other buffer stores.

[0116] At 830, the processing grid 210 optionally adds the at least one waiting thread to the plurality of concurrent threads for execution by the plurality of logic elements 211. Optionally, adding the at least one waiting thread to the plurality of concurrent threads includes storing one or more additional runtime context values ​​of the at least one waiting thread in the context storage 212. Storing the one or more additional runtime context values ​​of the at least one waiting thread in the context storage 212 facilitates execution of the at least one waiting thread in 612 of the method 600 above.

[0117] Optionally, at 830 , processing grid 210 loads one or more additional runtime context values ​​for the at least one waiting thread into one or more of the plurality of logic elements to initiate execution of the at least one waiting thread without using context storage 212 .

[0118] Optionally, in 821, processing grid 210 associates each waiting thread of at least one waiting thread (at 830, the at least one waiting thread will be added to the plurality of concurrent threads) with a context identification value. Optionally, the context identification value indicates a context of the waiting thread in context storage 212. For example, the context identification value may be a thread identification value of the waiting thread.

[0119] In some embodiments, some common resources may be used to manage each group of waiting threads in one or more groups of waiting threads of one or more waiting threads. In such an embodiment, the buffer scheduling circuit 412 is optionally configured to generate a common thread context associated with each waiting thread in the at least one group of waiting threads in the buffer storage device 410 for at least one group of waiting threads in the one or more groups of waiting threads. Optionally, the buffer scheduling circuit 412 is optionally configured to generate a common registration entry associated with each waiting thread in the at least one group of waiting threads in the registration circuit 411. Optionally, when a new thread in the at least one group of waiting threads is selected as at least one waiting thread in 820, the buffer scheduling circuit 412 is optionally configured to calculate one or more private context values ​​of the new thread. Optionally, the one or more private context values ​​are derived according to a row index in the buffer storage device 410 or another row index in the context storage device 212. Optionally, the one or more private context values ​​are derived according to a column index in the buffer storage device 410 or another column index in the context storage device 212. Optionally, the private value is a running counter that is sequentially incremented or decremented using a step value, optionally using a modulus operation. Optionally, the private value is computed using a hash function. Optionally, the private value is computed using a function related to data structure locality. Optionally, the private value is an identified value.

[0120] Descriptions of various embodiments have been given for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements over existing technologies on the market, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

[0121] It is expected that during the patent expiration period of this application, many related processing grids will be developed, and the scope of the term "processing grid" is intended to include all such prior new technologies.

[0122] As used herein, the term "about" refers to ± 10%.

[0123] The terms "including," "comprising," "having," and their conjugations mean "including but not limited to." This term encompasses the terms "consisting of" and "consisting essentially of.

[0124] The phrase "consisting essentially of" means that the composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially change the basic and novel characteristics of the claimed composition or method.

[0125] As used herein, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0126] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features of other embodiments.

[0127] The word “optionally” is used herein to mean “provided in some embodiments and not provided in other embodiments.” Any particular embodiment may include multiple “optional” features unless such features conflict.

[0128] Throughout the application, various embodiments may be presented in a range format. It should be understood that the description in a range format is only for convenience and brevity and should not be interpreted as a strict limitation on the scope of the embodiments. Therefore, the range description should be deemed to clearly disclose all feasible sub-ranges and single numerical values ​​within the range. For example, a range description, such as from 1 to 6, should be deemed to clearly disclose sub-ranges, such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and single numbers within the range, such as 1, 2, 3, 4, 5, and 6. This applies no matter how wide the range is.

[0129] Whenever a numerical range is indicated herein, it is meant to include any cited number (fractional or integer) within the indicated range. The phrases "ranging between" a first indicated number and a second indicated number and "ranging from" a first indicated number "to" a second indicated number are used interchangeably herein and are meant to include the first indicated number and the second indicated number and all fractions and integers therebetween.

[0130] It should be understood that certain features of embodiments described in the context of separate embodiments for the sake of clarity may also be provided in combination in a single embodiment. Conversely, various features of embodiments described in the context of a single embodiment for the sake of brevity may also be provided separately or in any suitable subcombination or in any other described embodiment in a suitable manner. Certain features described in the context of various embodiments should not be considered essential features of those embodiments unless the embodiment cannot function without those elements.

[0131] Although the embodiments have been described in conjunction with their specific embodiments, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Therefore, it is intended to encompass all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0132] Applicants intend that all publications, patents and patent applications mentioned in this specification are incorporated by reference in their entirety into this specification, just as each individual publication, patent or patent application is expressly and individually indicated as being incorporated by reference herein when cited. In addition, the citation or identification of any reference in this application should not be construed as an admission that such reference can serve as prior art to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting. In addition, any priority document of this application is incorporated by reference in its entirety herein.

Claims

1. A system for processing multiple concurrent threads, comprising: At least one reconfigurable processing grid comprising: a plurality of logic elements; and A context storage device is configured to store a plurality of thread contexts, each thread context corresponds to one of a plurality of concurrent threads, each concurrent thread implements a data flow graph including a plurality of operations, the plurality of operations including at least one identified operation, wherein each thread context of the plurality of thread contexts includes: for a concurrent thread corresponding to the thread context, at least one runtime context value of the data flow graph implemented by the concurrent thread; and at least one hardware processor configured to configure the at least one reconfigurable processing grid: executing a first thread of the plurality of concurrent threads; and While executing the first thread: storing at least one runtime context value of the first thread in the context storage device; executing, by at least one identified logic element of the plurality of logic elements, the at least one identified operation of a second thread of the plurality of concurrent threads while waiting for completion of execution of at least one identified operation of the plurality of operations of the first thread by at least one identified logic element of the plurality of logic elements; and When execution of the at least one identified operation by the first thread is completed: retrieving the at least one runtime context value for the first thread from the context storage; and At least one other operation of the plurality of operations of the first thread is performed.

2. The system according to claim 1, wherein: The context storage device includes a plurality of context entries, each of which is used to store a plurality of runtime context values ​​of a thread context among the plurality of thread contexts.

3. The system according to claim 2, wherein: For at least one thread of the plurality of concurrent threads, a plurality of runtime context values ​​for the at least one thread are stored in more than one context entry of the plurality of context entries.

4. The system according to claim 3, wherein: The plurality of context entries are organized in a table having a plurality of rows, each row corresponding to one of the plurality of context entries; and Each of the plurality of rows has a plurality of columns, such that each of the plurality of runtime context values ​​of the thread context stored in the row is stored in one of the plurality of columns.

5. The system according to claim 4, wherein: The data flow graph includes a plurality of nodes and a plurality of edges; wherein at least one of the plurality of nodes implements a lookup table; and Wherein, configuring the at least one reconfigurable processing grid to execute the first thread includes storing the lookup table in at least one other column of the plurality of columns.

6. The system according to claim 2, wherein: The at least one reconfigurable processing grid is further configured to: When execution of at least one identified operation of the first thread is completed: At least one result value is stored in a context entry of the plurality of context entries storing at least a portion of a thread context for the first thread, the at least one result value being a result of executing at least one identified operation of the first thread.

7. The system according to claim 2, wherein: A first context entry of the plurality of context entries stores a plurality of runtime context values ​​for the first thread; and Wherein, the at least one reconfigurable processing grid is further configured to: computing an indication that the first context is complete based on a result of at least one test applied to a plurality of runtime context values ​​of the first context entry; and According to the indication that the first context is complete, at least one runtime context value of the first thread is retrieved and the at least one other operation is performed.

8. The system according to claim 7, wherein: The at least one reconfigurable processing grid further includes scheduling circuitry for applying the at least one test to the plurality of runtime context values.

9. The system according to claim 8, wherein: Applying the at least one test to the plurality of runtime context values ​​includes the scheduling circuitry executing a set of test instructions.

10. The system according to claim 7, wherein: The at least one reconfigurable processing grid is further configured to: select the first thread to execute the at least one other operation among the plurality of operations of the first thread according to a scheduling policy.

11. The system according to claim 7, wherein: The at least one reconfigurable processing grid is further configured to: Based on a tag added to one or more context entries of the plurality of context entries for storing at least a portion of the thread context of the first thread, at least one of the following is performed: refusing to execute the at least one other operation of the plurality of operations of the first thread; as well as At least one of a plurality of thread context values ​​for the first thread is provided to at least one other software object.

12. The system of claim 1, wherein: The data flow graph includes a plurality of nodes and a plurality of edges; wherein the at least one identified operation is represented in the data flow graph by at least one identified node among the plurality of nodes; wherein the at least one hardware processor is further configured to: identify a residual subgraph in the data flow graph, such that the residual subgraph consists of a subset of nodes in the plurality of nodes and a subset of edges in the plurality of edges, wherein in the data flow graph, there is no path between any two nodes in the at least one identified node, wherein, for each node in the node subset, there is no path in the data flow graph between the at least one identified node and the node, and wherein, for each edge in the edge subset, there is no path in the data flow graph between the at least one identified node and the edge; and The at least one runtime context value is at least one edge value of at least one edge in the edge subset.

13. The system according to claim 12, wherein: The data flow graph is a directed graph; wherein each of the plurality of edges has a head node among the plurality of nodes and a tail node among the plurality of nodes; wherein the node subset includes one or more entry nodes of the residual subgraph, wherein the entry node is not a head node of any edge in the edge subset; and The at least one runtime context value is at least one input value of at least one entry node among the one or more entry nodes.

14. The system of claim 1, wherein: The at least one identified operation includes at least one of: a memory access operation, a floating point math operation, executing another computation graph, an access to a coprocessor, and an access to a peripheral device connected to the at least one reconfigurable processing grid.

15. The system of claim 1, wherein: The multiple concurrent threads are a subset of a concurrent thread set, and each concurrent thread in the concurrent thread set implements the data flow graph; The system further comprises: a buffer storage device for storing a plurality of other thread contexts, each of the plurality of other thread contexts corresponding to at least one concurrent thread in the concurrent thread set; and Wherein, the at least one hardware processor is further configured to further configure the at least one reconfigurable processing grid: storing in the buffer storage one or more additional runtime context values ​​for one or more waiting threads, wherein the one or more waiting threads are not members of the plurality of concurrent threads; and In each of the multiple iterations: identifying at least one additional thread from among the plurality of concurrent threads that has completed execution; For at least one waiting thread among the one or more waiting threads, retrieving at least one additional runtime context value for the at least one waiting thread from the cache storage; and The at least one waiting thread is added to the plurality of concurrent threads for execution by the plurality of logic elements.

16. The system according to claim 15, in, The at least one reconfigurable processing grid further comprises: register circuitry for tracking the one or more waiting threads; and Additional scheduling circuitry for managing the execution of the multiple concurrent threads; Wherein, the additional scheduling circuit is configured as follows: selecting the at least one waiting thread from the registration circuit; retrieving the at least one additional runtime context value for the at least one waiting thread from the cache storage; and The at least one waiting thread is added to the plurality of concurrent threads for execution by the plurality of logic elements.

17. The system of claim 16, wherein: Adding the at least one waiting thread to the plurality of concurrent threads includes storing the at least one additional runtime context value for the at least one waiting thread in the context storage.

18. The system of claim 16, wherein: The additional scheduling circuit is further configured to: Each of the at least one waiting thread is associated with a context identification value indicating a thread context of the waiting thread in the context storage device.

19. The system of claim 16, wherein: The buffer storage device comprises a plurality of buffer entries, each buffer entry being used to store a thread context of at least one waiting thread among the one or more waiting threads; Wherein, the registration circuit comprises a plurality of registration entries, each registration entry being used to track at least one of the one or more waiting threads; Wherein, the additional scheduling circuit is further configured as: For at least one group of waiting threads among the one or more waiting threads, generating in the buffer storage device a common thread context associated with each waiting thread among the at least one group of waiting threads; generating in the registration circuit a common registration entry associated with each waiting thread in the at least one set of waiting threads; and When a new thread in the at least one group of waiting threads is selected from the registration circuit as the at least one waiting thread, at least one private context value of the new thread is calculated.

20. The system of claim 16, wherein: The registration circuit is further configured to suspend execution of at least some of the concurrent threads in the set of concurrent threads until a number of the one or more waiting threads exceeds a threshold.

21. The system of claim 1, wherein: The plurality of logic elements are a plurality of reconfigurable logic elements organized into a plurality of computational groups; and Wherein, the at least one identified logic element is a subset of the plurality of computational groups.

22. The system of claim 1, wherein: The runtime context value of the data flow graph is an input value or an output value of a node among the plurality of nodes of the data flow graph.

23. The system of claim 1, wherein: The at least one hardware processor is further configured to: configure the at least one reconfigurable processing grid to execute the first thread in each thread iteration of a plurality of thread iterations; Wherein, the context storage device comprises: at least one additional context entry for storing additional multiple runtime context values ​​of the data flow graph, wherein the additional multiple runtime context values ​​are common to the multiple thread iterations; and Wherein, when execution of at least one identified operation of the first thread is completed, the at least one reconfigurable processing grid is further configured to retrieve at least one of the additional plurality of runtime context values ​​from the context storage device.

24. The system of claim 1, wherein: The at least one reconfigurable processing grid further comprises at least one other context storage device; and Wherein, the at least one hardware processor is further configured to configure the at least one reconfigurable processing grid: When execution of at least one identified operation of the first thread is completed: storing at least one additional runtime context value for the first thread in the at least one other context storage device; further executing the first thread; and While further executing the first thread: While waiting for at least one other identified operation of the plurality of operations of the first thread to be further performed by at least one other identified logic element of the plurality of logic elements to complete, the at least one other identified operation of another second thread of the plurality of threads is performed by the at least one other identified logic element.

25. The system of claim 1, wherein: The at least one reconfigurable processing grid further comprises at least one counter; wherein the plurality of concurrent threads comprises a concurrent thread group associated with the at least one counter; and The at least one runtime context value comprises at least one counter value read from the at least one counter by accessing the at least one counter.

26. The system of claim 25, wherein: Each concurrent thread in the concurrent thread group implements the identified dataflow graph.

27. The system of claim 25, wherein: Accessing the at least one counter comprises an atomic access comprising reading the at least one counter and incrementing the at least one counter.

28. The system of claim 27, wherein: The at least one counter is a sequence of counters; and Wherein, incrementing the at least one counter comprises at least one of the following: incrementing a first counter in the sequence of counters using modulo arithmetic and, if the first counter wraps around after being incremented, incrementing a second counter in the sequence of counters that is consecutive to the first counter; and Incrementing a first counter in the sequence of counters, and if the first counter exceeds a maximum value: incrementing a second counter in the sequence of counters that is consecutive to the first counter; and At least one of the following: setting the first counter to a new value calculated using the second counter; and The maximum value is set to another new value calculated using the second counter.

29. A method for processing multiple concurrent threads, comprising: executing a first thread of a plurality of concurrent threads, each concurrent thread implementing a data flow graph including a plurality of operations, the plurality of operations including at least one identified operation; as well as While executing the first thread: storing at least one runtime context value of the data flow graph implemented by the first thread in a context storage device, wherein the context storage device is configured to store a plurality of thread contexts, each thread context corresponding to one of the plurality of concurrent threads, wherein each thread context in the plurality of thread contexts comprises: for a concurrent thread corresponding to the thread context, at least one runtime context value of the data flow graph implemented by the concurrent thread; executing, by at least one identified logic element of the plurality of logic elements, the at least one identified operation of a second thread of the plurality of concurrent threads while waiting for completion of execution of at least one identified operation of the plurality of operations of the first thread by at least one identified logic element of the plurality of logic elements; and When execution of the at least one identified operation by the first thread is completed: retrieving the at least one runtime context value for the first thread from the context storage; and At least one other operation of the plurality of operations of the first thread is performed.

30. A software program product for executing multiple concurrent threads, comprising: a non-transitory computer-readable storage medium; first program instructions for executing a first thread of a plurality of concurrent threads, each concurrent thread implementing a data flow graph including a plurality of operations including at least one identified operation; as well as The second program instruction is configured to: while executing the first thread: storing at least one runtime context value of the data flow graph implemented by the first thread in a context storage device, wherein the context storage device is configured to store a plurality of thread contexts, each thread context corresponding to one of the plurality of concurrent threads, wherein each thread context in the plurality of thread contexts comprises: for a concurrent thread corresponding to the thread context, at least one runtime context value of the data flow graph implemented by the concurrent thread; executing, by at least one identified logic element of the plurality of logic elements, the at least one identified operation of a second thread of the plurality of concurrent threads while waiting for completion of execution of at least one identified operation of the plurality of operations of the first thread by at least one identified logic element of the plurality of logic elements; and When execution of the at least one identified operation by the first thread is completed: retrieving the at least one runtime context value for the first thread from the context storage; and executing at least one other operation of the plurality of operations of the first thread, Wherein, the first program instructions and the second program instructions are executed by at least one computerized processor from the non-transitory computer-readable storage medium.

Images