Method, processor, apparatus and program product for processing instruction unit

Abstract

Embodiments of the invention relate to a method for processing an instruction unit, a processor, an electronic device, and a computer program product. The method comprises the steps that a first thread running on a processor executes a first execution step on an instruction unit within a first time period; the method further includes executing, by a second thread running on the processor, a second execution step of the set of execution steps on the instruction unit for a second time period. The first execution step and the second execution step are from a group of execution steps, and the second time period is at least partially overlapped with the first time period. In this way, the execution steps of the instruction units can be executed in parallel, and therefore the execution time can be shortened, and available threads can be utilized more densely and more evenly.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of computer technology, and more particularly to methods for processing instruction units, processors, electronic devices, and computer program products. Background Technology

[0002] A dataflow architecture is a computer architecture that represents a program as a dataflow graph (DFG). In a DFG, nodes are "operators," and the directed edges between these nodes are "channels" through which immutable values ​​are transferred between nodes. Conceptually, once data is available on all input channels of an operator, the operator can be evaluated. After evaluation, the operator obtains the result data at the end of its output channel, which can be used as input to evaluate other nodes. During the execution of a DFG, the order in which operators are evaluated is undefined.

[0003] In a computer with a data stream processor, once it is determined that an operator has all available input data, a thread can fetch the corresponding instruction unit from memory and compute the instruction unit to generate a result token. The result is then passed to other instruction units that expect the result token as input for evaluation. Summary of the Invention

[0004] Typically, embodiments of the present invention provide a scheme for optimizing computation graph compilation.

[0005] In a first aspect, a method is provided. The method includes: a first thread running on a processor executing a first execution step of a set of execution steps on an instruction unit during a first time period; and a second thread running on the processor executing a second execution step of the set of execution steps on the instruction unit during a second time period, wherein the second time period at least partially overlaps with the first time period. In this manner, the execution steps for the instruction unit are treated as atomic tasks and can be executed in parallel, thereby reducing execution time and making more intensive and balanced use of available threads.

[0006] In some embodiments of the first aspect, the method further includes: loading the instruction unit and its metadata into memory, wherein the metadata indicates one of a plurality of execution states. The metadata can be used to determine the state of the instruction unit in the execution workflow, which provides a basis for parallel execution steps.

[0007] In some embodiments of the first aspect, executing the first execution step on the instruction unit includes: based on the metadata of the instruction unit, transitioning the instruction unit from a first execution state among a plurality of execution states to a second execution state among the plurality of execution states. In this way, each step of executing an instruction unit can be achieved through transitions between non-binary states of the unit.

[0008] In some embodiments of the first aspect, transitioning the instruction unit from the first execution state to the second execution state includes: updating the metadata of the instruction unit based on the execution of the first execution step to indicate the second execution state. In this way, the execution of another step by a thread can be determined.

[0009] In some embodiments of the first aspect, the method further includes: the first thread selecting an instruction unit from a set of instruction units based on the metadata of the instruction unit, for executing the first execution step. In this way, a thread can capture a suitable unit to execute, regardless of whether that unit is being processed by another thread.

[0010] In some embodiments of the first aspect, selecting the instruction unit includes determining that the metadata of the instruction unit satisfies the requirements for performing the first execution step. In this way, the thread can determine the appropriate steps to execute on the captured unit.

[0011] In some embodiments of the first aspect, the metadata of the instruction unit indicates that one or more inputs to the instruction unit are being evaluated, wherein performing the first execution step on the instruction unit includes: evaluating the instruction unit based on the one or more inputs in response to determining that the one or more inputs are available. In this way, the thread can determine an appropriate opportunity to perform the evaluation of the instruction unit.

[0012] In some embodiments of the first aspect, evaluating the instruction unit includes: performing an evaluation using one or more inputs corresponding to operations of the instruction unit; and storing the output of the evaluation in the metadata of the instruction unit. In this way, evaluation and delivery can be separated into different execution steps, thereby refining the granularity of atomic steps in the execution workflow.

[0013] In some embodiments of the first aspect, the metadata of the instruction unit indicates the output for evaluating the operation corresponding to the instruction unit, and performing the first execution step on the instruction unit includes: determining one or more instruction units from the set of instruction units in response to determining that the output is available in the metadata; and storing the output in the metadata of the one or more instruction units as input for evaluating the one or more instruction units. This provides a way to perform a pass-through step suitable for the step-level penalty method disclosed herein.

[0014] In some embodiments of the first aspect, each instruction unit in the set of instruction units corresponds to a node in the dataflow graph (DFG) of the program to be executed. In this way, step-level parallelism can be applied to the program to be executed, thereby improving program execution performance.

[0015] In some embodiments of the first aspect, the method further includes: a third thread running on the processor executing a third execution step of the set of execution steps on the instruction unit during a third time period; and locking at least the instruction unit according to the third execution step during the execution of the third execution step. This gradual locking strategy improves thread safety while avoiding locking only one unit for one thread throughout the entire workflow.

[0016] In some embodiments of the first aspect, the method further includes: a fourth thread running on the processor simultaneously executing the same type of execution steps on multiple instruction units. In this way, the overall utilization of the threads can be reduced, and idle time can be used for other processing, while still maintaining the advantages of step-level parallelism.

[0017] In a second aspect, a processor is provided, the processor being configured to: execute a first execution step of a set of execution steps on an instruction unit during a first time period via a first thread running on the processor; and execute a second execution step of the set of execution steps on the instruction unit during a second time period via a second thread running on the processor, wherein the second time period at least partially overlaps with the first time period.

[0018] Thirdly, an electronic device is provided. The electronic device includes: a processor according to a second aspect of the invention and a memory coupled to the processor. The memory stores instruction units executed by the processor.

[0019] A computer program product is provided, which is tangibly stored on a computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to perform the method according to a first aspect of the invention.

[0020] It should be understood that the summary section is not intended to identify key or essential features of the embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] Some embodiments will now be described with reference to the accompanying drawings, in which: Figure 1 A schematic diagram of an exemplary environment in which various embodiments of the present invention may be implemented is shown; Figure 2 A flowchart illustrating an exemplary method for processing instruction units provided by some embodiments of the present invention is shown; Figure 3 An exemplary schematic diagram of the decomposition and parallel execution steps of a workflow for executing instruction units provided by some embodiments of the present invention is shown; Figure 4 The present invention illustrates a conceptual architecture of an instruction unit for executing a program, provided by some embodiments thereof; Figure 5 Exemplary DFGs representing executable programs, provided by some embodiments of the present invention, are illustrated. Figure 6 An exemplary timeline of the execution of a program by multiple threads running on a processor is shown, wherein the steps of an execution unit are executed sequentially by a single thread; Figure 7 The diagram illustrates an example of the execution process of an instruction unit of a multi-threaded program according to some embodiments of the present invention. Figure 8 Another example diagram illustrates the execution process of instruction units of a multi-threaded program provided in some embodiments of the present invention; Figure 9 A schematic block diagram of a device that can be used to implement embodiments of the present invention is shown.

[0022] In all the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0023] The principles of the invention will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement the invention, and do not impose any limitation on the scope of the invention. The disclosure described herein can be implemented in various ways other than those described below.

[0024] In the following detailed description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] In this invention, references to "one embodiment," "some embodiments," "embodiment," etc., indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment necessarily includes such specific features, structures, or characteristics. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with some embodiments, it is assumed that the effect of such feature, structure, or characteristic on other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0026] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element without departing from the scope of the embodiments, and similarly, a second element may be referred to as a first element. As used herein, the term “and / or” includes any and all combinations of one or more of the listed items.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments. Unless the context clearly indicates otherwise, the singular forms “a” and “described” as used herein are also intended to include the plural forms. It should also be understood that the terms “comprising,” “having,” and / or “including,” when used herein, specify the presence of said features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0028] In a dataflow architecture, the program to be executed is loaded into memory as a set of instruction units, each corresponding to a node in the Data Flow Group (DFG) representing the program. Conceptually, when the processor executes a unit of the program, an "arbitration network" captures that unit and checks if all of its inputs have available data so that it can be evaluated. The unit is then sent to an arithmetic unit, such as an arithmetic logic unit (ALU), for evaluation. Once the ALU has computed the evaluation result for the unit, an "assignment network" finds one or more units that need the result and stores the result in the found unit.

[0029] Although the exact order of steps may differ across different processor implementations, the execution workflow of individual instruction units is sequential in these implementations. Once an instruction unit to be executed is captured, one thread of the processor locks and holds that unit until the evaluation and allocation of the result are complete. The unit must traverse the entire workflow, while other idle threads must wait until any unit becomes available for execution, which can limit thread utilization.

[0030] Furthermore, due to varying complexity, the durations of different steps are not uniform. For example, checking the input availability of a unit is much faster than multiplying two floating-point numbers. Additionally, the cost of ALU evaluating different units (e.g., in CPU ticks) can also vary significantly. For this reason, wait times may not be evenly distributed across threads, potentially leading to suboptimal execution performance.

[0031] Some approaches attempt to improve the performance of such processors through hardware-level task schedulers, thread criticality predictors, and / or thread accelerators. However, these approaches focus on thread-level parallelism, where a single unit of instruction is treated as an atomic job.

[0032] According to embodiments of the present invention, a method is provided for processing instruction units of a program in a dataflow machine. This method supports step-level parallelism (SLP), enabling the parallel execution of multiple steps of an execution workflow on a unit. Specifically, when a thread running on the processor executes one execution step on an instruction unit within a first time period, another thread running on the processor can execute another execution step on the same instruction unit within a second time period, the second time period at least partially overlapping with the first time period. In this way, execution time can be reduced, and available threads can be utilized more intensively and evenly, thereby improving execution performance.

[0033] The principles and embodiments of the present invention will now be described in detail with reference to the accompanying drawings. First, refer to... Figure 1 , Figure 1 A schematic diagram of an exemplary environment 100 in which various embodiments of the present invention may be implemented is shown.

[0034] Environment 100 includes computing device 110. Examples of computing device 110 include, but are not limited to, desktop computers, smartphones, tablets, laptops, and server computers. The scope of the invention is not limited thereto. Computing device 110 includes processor 120 and memory 130 coupled to processor 120, which may be, for example, random access memory (RAM).

[0035] The computing device 110 can use the processor 910 to execute various programs of the embodiments of the present invention. In order to execute a program, the program can be parsed into a DFG representation and loaded into the memory 130 as a set of instruction units 140-1, 140-2, ..., 140-N (collectively referred to as instruction units 140). Each instruction unit corresponds to a node of the DFG, that is, the basic operation of the program to be executed.

[0036] The computing device 110 can launch multiple threads 150-1, 150-2, ..., 150-N (collectively referred to as threads 150) on the processor 910 to execute the program. As those skilled in the art know, threads can divide the physical core of the processor into multiple virtual cores, thereby enabling parallel execution. During execution, each thread can capture an available unit for execution.

[0037] The execution workflow of a unit includes multiple steps. In this embodiment of the invention, when one of the multiple threads 150 needs to capture a unit to perform a step on it, another thread of the multiple threads 150 can capture the same unit in parallel to perform another step. Therefore, during execution, when there are n instruction units available for parallel execution according to the DFG of the program being executed, more than n threads may be occupied by a unit, some of which are processing the same instruction unit.

[0038] It should be understood that environment 100 is described for illustrative purposes, and other devices, systems, or components not shown in environment 100 may also exist. For example, computing device 110 may start another set of threads to execute another program. For example, computing device 110 may have a coprocessor that can execute programs in a manner similar to processor 130.

[0039] Now for reference Figure 2 , Figure 2 A flowchart is shown of an exemplary method 200 for processing instruction units provided by some embodiments of the present invention. This exemplary method 200 may, for example, be derived from... Figure 1The computing device 110 shown performs the operation. It should be understood that method 200 may also include other actions not shown. The scope of the invention is not limited thereto. The following description, in conjunction with... Figure 1 The exemplary environment 100 describes method 200 in detail.

[0040] 210, a first thread running on the processor executes a first execution step on an instruction unit within a first time period. The first execution step comes from a set of execution steps. The processor's execution workflow for instruction units may include this set of steps. For example, computing device 110 may cause thread 150-1 to execute one execution step on instruction unit 140-1 within the first time period.

[0041] Each instruction unit can correspond to a node in the Data Flow Group (DFG) of the program to be executed. A set of instruction units of the program can be loaded by computing device 110 into memory 130 for execution, following the principles of a dataflow machine. In some embodiments, each instruction unit may be loaded with additional metadata indicating that the unit is in one of a plurality of execution states. Thread 150-1 can then select an instruction unit for execution steps. In some embodiments, thread 150-1 can determine that the metadata of an instruction unit satisfies the requirements for executing a certain execution step. Once determined, thread 150-1 can select the instruction unit and execute the corresponding step.

[0042] 220. During a second time period, a second thread running on the processor executes a second execution step on an instruction unit 140-1, the second time period at least partially overlapping with the first time period. The second step is another step in a set of execution steps in an execution workflow. For example, computing device 110 may cause thread 150-2 to execute another execution step on instruction unit 140-1, while thread 150-2 still executes that execution step on instruction 140-1 at 210. In different examples, the second time period may start earlier than, later than, or simultaneously with the first time period. Furthermore, the second time period may end earlier than, later than, or simultaneously with the first time period.

[0043] Similar to thread 150-1, once thread 150-2 determines that the metadata of instruction unit 140-1 meets the requirements for executing another execution step (e.g., all prerequisites are met and there are no other restrictions), it can select instruction unit 140-1 to execute another step on it, regardless of whether thread 150-1 is still processing instruction unit 140-1, as in 210. In some embodiments, a unit can be captured in parallel by two or more threads, each thread can execute one execution step on it. The steps that can be executed in parallel vary in different implementations, which will be described in more detail below.

[0044] Method 200 allows certain steps for executing the same instruction unit to be performed in parallel by multiple threads, thereby reducing execution time and making more intensive and balanced use of the available threads for executing the instruction unit of the program.

[0045] Figure 3 A schematic diagram 300 illustrates the breakdown and parallel execution steps of a workflow for executing an instruction unit provided by the present invention. For comparison, Figure 300 presents two exemplary processes for executing the same instruction unit on timeline 330, wherein each step in the execution workflow is represented as a rectangle. It should be understood that Figure 300 is presented in a qualitative manner and may not be quantitatively precise. Furthermore, the number of steps is merely illustrative, and the number of steps in the execution workflow may differ in different implementations.

[0046] As illustrated in exemplary process 310, the processor's execution workflow for a single instruction unit in this example comprises a set of steps 311 to 314. The workflow completes at time t2 333 when the thread running on the processor begins to execute the unit step by step at time t0 331.

[0047] Depending on their nature and specific implementation, these steps are partially ordered. In other words, some steps must be executed strictly in sequence, while others are independent of each other's results and can therefore be completed in parallel if other prerequisites for execution are met. For example, steps 312 and 313 are independent of each other, and each of them can be executed after step 311 is completed. For example, in a dynamic dataflow architecture, the marking and evaluation steps of a single unit can be executed in parallel.

[0048] Therefore, step 313 does not need to wait for step 312 to complete. According to some embodiments of the invention, exemplary process 320 presents another execution of the same instruction unit by the processor. For simplicity, it is assumed that the time taken for each step is the same as in process 310, and process 320 also begins at time t0 331. The difference is that in process 320, steps 322 and 323 are parallel. However, process 320 can complete at time t1 332, which is significantly earlier than time t2 333.

[0049] To illustrate how to execute steps in parallel, refer to [reference needed]. Figure 4 This illustrates an exemplary conceptual architecture 400 of instruction units for executing a program provided by some embodiments of the present invention. For example, architecture 400 can be... Figure 1 It is implemented on computing device 110.

[0050] In architecture 400, a set of instruction units of a program to be executed can be loaded into memory 430, such as unit 431. Each unit corresponds to a node in the program's DFG. Arbitration network 410 can receive "operation packets" from the satisfied units for execution. Threads running on the processor can execute actions of arbitration network 410, which captures a unit from the set of instruction units for execution.

[0051] When a thread in the arbitration network 410 captures a unit, it can execute one step of the workflow instead of going through the entire execution workflow of the captured unit. Conceptually, after executing the corresponding execution step, the process executed by the thread can be performed by the unit allocation network 420. The thread can perform the actions of the allocation network 410, which modifies the unit as necessary based on the executed steps. The thread can then capture another available unit for performing a step on that available unit.

[0052] In architecture 400, the process can flow to check step 441, where the thread checks the captured unit to determine if it is ready for evaluation. Depending on the implementation, a unit determined to be ready may be captured by the same thread in the next round, or it may be captured by a different thread to perform another step.

[0053] In architecture 400, when a cell is ready for evaluation, the process can flow to evaluation step 442. In this step, a thread can send the captured cell's operation packet 452 to an arithmetic unit (e.g., the ALU in the processor). The arithmetic unit can compute the evaluation result of the operation corresponding to the instruction unit, which transforms the operation packet 452 into a data packet 451 containing the evaluation result. Unlike conventional dataflow machines, the evaluation result can initially be stored with the cell, for example, in the metadata 432 of cell 431, as will be described in more detail below.

[0054] In architecture 400, for a cell that has obtained an evaluation result, the process can proceed to step 443. In this step, the result data is retrieved from the cell and passed to one or more instruction cells that are waiting for the result data. Threads can execute the actions of allocation network 420, which places data or references to data into the corresponding instruction cell.

[0055] In architecture 400, the process can also flow to a marking step 444, for example, in the case of a dynamic dataflow architecture, during which captured units are marked. Furthermore, for instruction units, architecture 400 supports the parallel execution of evaluation 442 and marking 444 for the same instruction unit. For example, while one thread captures unit 431 and is still undergoing the marking process 444, another thread can capture unit 431 and undergo the evaluation process 442 in parallel. It should be understood that... Figure 4 The specific steps shown are merely illustrative. According to the present invention, the steps of the workflow for executing instruction units and the steps that can be parallelized may vary in different implementations.

[0056] In order for threads to capture units to execute appropriate steps, in architecture 400, each instruction unit can load the unit's metadata (e.g., metadata 432 of unit 431). Therefore, each unit has additional read / write memory space for maintaining metadata about the unit.

[0057] The metadata of a unit can indicate one of several execution states. When an instruction unit is executed by a thread of a processor on a device implementing Architecture 400, the instruction unit can be in one of several execution states. Conversely, when the execution of an instruction unit is viewed as an atomic job in a regular machine, the instruction unit may only have a binary execution state to indicate whether the unit has been executed.

[0058] Based on the metadata of an instruction unit, an idle thread can select an available unit from a set of instruction units to execute the corresponding step. In some embodiments, when a thread determines that the metadata of a unit meets the requirements for executing the corresponding execution step, it selects (i.e., captures) the unit for execution. In some embodiments, for a selected unit, the thread can transition the unit from its current state to another state based on the unit's metadata. Based on the execution step, the thread can update the unit's metadata to indicate other execution states, thereby enabling the unit to be captured for the next execution step.

[0059] In this way, conceptually, each unit is transformed into a finite state machine (FSM) with multiple possible states corresponding to the steps in executing the workflow. Each thread can acquire a unit (or multiple units, as described later) and attempt to transition that unit to one of the next possible states.

[0060] Metadata can indicate one or more inputs used to evaluate a unit. For example, input data or references to data can be stored in slots within the metadata. In some embodiments, the metadata can indicate that the unit has all available inputs and that the unit is ready for evaluation. A thread can then perform an evaluation step on the unit based on one or more inputs. In the evaluation step, the thread can evaluate the operation corresponding to the instruction unit using one or more inputs. For example, it can send an operation packet of the unit with the inputs to the ALU so that the ALU computes the output of the operation. In some embodiments, the metadata of the instruction unit can indicate the output of the operation corresponding to the instruction unit. In the evaluation step, the thread can update the metadata to store the evaluation result in the metadata, for example, in the corresponding slot.

[0061] In some embodiments, metadata may indicate that a unit has been evaluated and its output is available in its metadata, thus the unit is ready for the delivery step. In this step, a thread can determine one or more instruction units from a set of instruction units. For example, the metadata may store dependencies between instruction units based on the DGF of the executing program, and as a function of the assignment step, the thread can determine one or more units waiting for the output of the current unit. The thread can then store the output as input in the metadata of one or more waiting units.

[0062] The state corresponding to a particular step can include several aspects, such as whether the state indicates no conflict with other threads performing that specific step. This can be supported by some step-locking mechanism, which will be described later. For example, in a case where the marking and evaluation of a cell can be performed in parallel, if all inputs are available and the cell is being marked or is being marked by another thread, that thread can perform the evaluation step but not the marking step.

[0063] In addition to the states and corresponding steps described above, there may be other states and corresponding steps. For example, the state may indicate that the unit is still waiting for one or more of its inputs, and that the thread or other threads will check the state again later. For example, the state may indicate that the unit's output has been assigned to another unit, and the thread may perform a deletion step to remove the unit from memory or otherwise make it unavailable.

[0064] For ease of explanation, the states before and after executing a set of exemplary steps in an execution workflow are presented in mathematical form below. In the mathematical form, instruction units are defined as sets, as shown in formula (1): (1) in, v This refers to the operators in DFG, with an element number of mψ refers to the "parent" cell that needs to be evaluated in order to evaluate the current cell. k j It is an evaluation slot associated with an edge of v, as shown in (2): (2) in" "This indicates that they are not yet interested in assessing that side," "This indicates that the edge may need to be evaluated." "Indicates that it must be used with" β Operators for parent units v Assess this side, "This indicates that the data for this side will be immediately available after the evaluation." β.α Read from slot, "Indicates data" d It is already in the cell and can be propagated to other waiting cells.

[0065] Based on this, a set of exemplary steps in the execution workflow are given in the following formulas (3) to (7), wherein the sub-formulas above the center line show the state of the unit that meets the requirements for executing the corresponding step, and the sub-formulas below the center line show the state that the unit needs to be transformed into by executing the steps.

[0066] Formula (3) presents the unit The "copy" step. By performing this step, the cell's input is copied into one or more of its input slots.

[0067] (3) Formula (4) presents the unit The "transfer" step. By performing this step, data is transferred from the unit... Passed to the waiting unit .

[0068] (4) Formula (5) presents the unit The "delete" step. By performing this step, the unit... It will be removed from memory or made unavailable.

[0069] (5) Formula (6) presents the “evaluation” step. By performing this step, the unit… The output is calculated and stored together with the cell.

[0070] (6) Formula (7) presents the unit The "instantiation" step. By performing this step, units can be instantiated as needed. .

[0071] (7) During the demonstration simulation of the program based on the above formula, the number of CPU ticks required to calculate the 16th Fibonacci number was 1.6 times that required to calculate the same Fibonacci number using the same simulator without parallel execution steps.

[0072] Understandably, the examples above are merely illustrative. The possible execution states of an instruction unit can vary depending on the arithmetic unit, arbitration network, and allocation network implemented in a particular processor. The transfer of "control" between steps is coordinated by the arbitration network. Its threads "activate" units to transition to the next state as necessary and possible.

[0073] The execution process according to an embodiment of the present invention will now be described with reference to the simplified exemplary procedure in the following text. This non-limiting exemplary procedure calculates the sum of two numbers. One number is calculated using a division operator, and the other number is calculated using a multiplication operator. Mathematically, the procedure calculates y in the following equation: Figure 5 An exemplary DFG500, representing an executable program as provided in some embodiments of the present invention, is shown. For example... Figure 5 As shown, DFG 500 consists of three nodes (vertices) 510, 520, and 530, each node representing an operator in the program. All necessary data can be provided as constants. Data constants are transmitted between operators via edges representing data channels.

[0074] Each operator has one or more input edges indicating the data required to evaluate the operator. The operator also has one or more output edges indicating which other operator the evaluation result should be passed to. For example, the evaluation of the division operator (“div”) represented by node 520 requires data 541 (“…”). ") and data 542 (" The input is shown by edges 551 and 552. Its evaluation result needs to be transmitted to the addition operator (“add”) represented by node 530 as the evaluation input, as shown by edge 553.

[0075] In summary, the program must evaluate y represented by data 543 and present it as the output of node 530. Among these operators, the evaluations of the operators represented by nodes 510 and 520 (i.e., "div" and "mul") do not depend on each other. Therefore, their evaluations can be parallel. On the other hand, the evaluation of the operator represented by node 530 must wait until the evaluation results of the other two operators are available.

[0076] In comparison, Figure 6 Figure 600 illustrates an example of the execution process of a program corresponding to the DFG 500, executed by multiple threads running on the processor. Figure 6 In the example, the steps of the execution unit are executed sequentially by a single thread.

[0077] Figure 6 The example illustrates the execution of DFG 500 by four threads: 610 (“T1”), 620 (“T2”), 630 (“T3”), and 640 (“T4”). Each thread is represented by a sequence of horizontal rectangles on timeline 650. Each rectangle in the sequence represents an execution step of an operator (the instruction unit corresponding to the operator) and occupies a certain amount of horizontal space, reflecting the time taken to execute that step. The horizontal space is measured in discrete increments, each representing a CPU tick, as shown on timeline 650.

[0078] It is understood that the duration of the steps is merely illustrative. Furthermore, for readability, a tick interval is shown between two consecutive rectangles. Those skilled in the art will understand that such an interval does not necessarily exist in practice.

[0079] like Figure 6 As shown, thread 610 can begin by checking step 611 (labeled "C") and can capture the first operator with all the necessary input data. In this example, it can capture the operator with... Figure 5 The instruction unit corresponding to the multiplication operator represented by node 510. The execution of this instruction unit is treated as an atomic job, and thread 610 can continue until the process is complete. In this illustrative example, thread 610 can first proceed to the marking step 612 (labeled "T") of the marked unit. Then, it can perform the evaluation step 613 (labeled "E") on the unit. mul The process involves calculating the result of an arithmetic multiplication. Afterward, thread 610 can execute pass step 614 (labeled "D"), passing the result of step 613 to the waiting units, which will be... Figure 5 The element corresponding to the addition operator shown in node 530.

[0080] Thread 620 can capture the instruction unit corresponding to the division operator represented by node 510. It follows the same route as thread 610 in processing units used for multiplication. However, as in evaluation step 623 (labeled "E") div As shown in the figure, due to the different complexities of the evaluation, the evaluation of the division takes longer than that of evaluation step 613. Therefore, the execution of the division in thread 620 is completed at a later time.

[0081] While thread 620 is still processing the division operator, thread 610 can check another instruction unit in check step 615 to attempt to capture it for processing. However, the only remaining unit (i.e., the unit corresponding to the addition operator) requires the evaluation result of the division operator as input and is not yet ready. Therefore, the check step will not allow any unit to be processed.

[0082] Subsequently, thread 610 may execute another checking step 616, which captures the instruction unit corresponding to the addition operator for further processing. During this program execution, the other two available threads 630 and 640 remain inactive. They attempt to check the instruction unit and capture available units (e.g., as shown in steps 631 and 641), but consistently fail.

[0083] exist Figure 6 In the coarse-grained parallel example shown, the execution workflow of an instruction unit is treated as an atomic job, executed sequentially by a single thread. In this case, the maximum number of threads the processor can utilize is equal to the number of operators that can be evaluated in parallel according to the DFG of the executing program. Adding more threads to the processor will not speed up the execution of this program.

[0084] Figure 7 Figure 700 illustrates an example of the execution process of instruction units of a multi-threaded executable program provided in some embodiments of the present invention. For illustration, Figure 7 The example illustrates the execution process of the DFG 500, which is also executed by four threads: threads 710 (“T1”), 720 (“T2”), 730 (“T3”), and 740 (“T4”). These threads are executed in conjunction with… Figure 6 The example in the example uses a similar format, which is presented on timeline 750.

[0085] exist Figure 7 In the example, a step in the execution workflow of a single instruction unit is treated as an atomic job, rather than as... Figure 6 The example shown is the entire workflow. Furthermore, some steps in the workflow are independent of each other and can therefore be executed in parallel. For illustration, in the example used for execution... Figure 7 In an exemplary implementation of the process, the marking step and the evaluation step for executing the same instruction unit can be executed in parallel.

[0086] like Figure 7 As shown, thread 710 can begin by checking step 711 and can capture the instruction unit corresponding to the multiplication operator (“mul”). In this example, thread 710 retains this capture to execute the marking step 712. On the other hand, thread 730 may not be able to... Figure 7 The inspection step 731 determines the inspection result of any unit, since the two units available at this time are inspected by threads 710 and 720.

[0087] Once it is determined in check step 711 that the unit corresponding to multiplication is available for execution, that unit can be used for the marking and evaluation steps. Therefore, while thread 710 executes the marking step 712, thread 730 can also pick up the unit for multiplication to perform the evaluation step 732 on that unit. Thus, the time periods for marking step 712 and evaluation step 732 for executing the same instruction unit can at least overlap. Similarly, thread 740 can pick up the unit corresponding to the division operator to perform the evaluation step 742 for division, while that unit is still marked by thread 720 in marking step 722.

[0088] In this example, after evaluation steps 732 and 742, threads 730 and 740 can retain the captured corresponding units and can further execute pass-through steps 733 and 743. Each of pass-through steps 733 and 743 passes the corresponding evaluation result to one or more waiting units, which are the units corresponding to the addition operator in this example.

[0089] After all the required inputs have been passed to the unit used for addition, thread 710 can perform a check step 716 on that unit to determine if it can be further processed. Then, thread 710 can perform a marking step 717 on that unit. Similarly, while the unit used for addition is still marked by thread 710, thread 720 can also capture that unit to perform an evaluation step 717 (labeled "E"). add (”).

[0090] Step-level fine-grained parallelism can speed up program execution and make more intensive use of available threads. For example, with... Figure 6 Compared to the process shown, Figure 7 The process shown reduces the time required to execute the same exemplary procedure from 32 ticks to 26 ticks (including exemplary pause ticks between steps).

[0091] Thread utilization can be viewed as the ratio between the CPU ticks used by a single thread and the total CPU ticks used during execution. Figure 6 In the example, the overall utilization of the thread pool was 36%, while Figure 7 In this configuration, thread pool utilization increases to 45% (including exemplary pause ticks when a unit is held by a thread, for example, as shown by arrow 760). Higher utilization allows for more efficient use of processor resources, including greater power efficiency.

[0092] This fine-grained parallelism allows for a more balanced utilization of all threads. For example, in Figure 6 In the coarse-grained example, while thread 610 is occupied most of the time, threads 630 and 640 remain inactive for the entire program's lifetime. In contrast, Figure 7 The four threads in the model are used in a more balanced manner, from 710 to 740.

[0093] It should be understood that Figure 700 is merely illustrative. For example, the thread that captures a certain step of a unit may differ in different executions of the program. In another execution, it could be one of threads 720 to 740 that performs the marking step for the unit used for addition, while thread 710 may perform the evaluation step in parallel.

[0094] In addition, Figure 7 In the example, after capturing a unit, a thread can retain the captured unit to execute the next possible step. This is merely illustrative, and various variations in the parallel execution steps of the same instruction unit are also within the scope of this invention. For example, after determining that a unit is available for execution, one thread can continue with the evaluation step for that unit, while another thread can capture and mark the unit in parallel. As another example, for each different execution step, the unit may be captured by a different thread. In other implementations, the execution process of an instruction unit may include more or fewer steps than those presented in the example, and / or steps different from those presented in the example, and these different steps can be executed in parallel in other implementations.

[0095] In some embodiments, at least some steps in the workflow can be implemented as parallel computations (also known as "matrix" or "vector" operations), meaning they modify multiple instruction units simultaneously, rather than one unit at a time. In other words, during program execution, a thread can capture multiple instruction units simultaneously and execute them. For example, a thread can determine that the metadata of each of the multiple instruction units satisfies the requirements for performing a specific execution step. After determination, the thread can capture multiple units to execute the specific execution step simultaneously.

[0096] Figure 8 Figure 800 illustrates another example of the process by which multiple threads execute instruction units of a program, as provided in some embodiments of the present invention. Figure 8The example illustrates the execution process of four threads, 810 (“T1”), 820 (“T2”), 830 (“T3”), and 840 (“T4”), on the DFG 500, and compares it with... Figure 6 and Figure 7 The examples in the text use a similar format to present them on timeline 850.

[0097] Used for execution Figure 8 The exemplary implementation of execution in [the code] supports the parallelism of the marking and evaluation steps for executing a unit of instruction. [This is related to] the execution of [the code]. Figure 7 The exemplary implementation shown differs from the exemplary execution. Figure 8 The exemplary implementation used also supports threads executing multiple instruction units' check and mark steps simultaneously.

[0098] As shown in the figure, thread 810 can check two available instruction units corresponding to the multiplication and division operators (represented by nodes 520 and 510, respectively). These two units can then be used for the marking and evaluation steps. Thread 810 can perform the marking step on the unit for multiplication and the unit for division. While thread 810 is still marking these two units, thread 830 can perform evaluation step 832 on the multiplication unit, and thread 830 can perform evaluation step 842 on the multiplication unit. Then, after the evaluation results of these two units have been transferred to the unit for addition, that unit becomes available and is checked by thread 810 in the checking step 816. Afterward, the marking step 817 and evaluation step 827 can be performed in parallel on the addition unit. In this example, no other units are executed while the unit for addition is being executed.

[0099] Because it supports processing the same step simultaneously on multiple units, it reduces the overall utilization of threads and allows idle time to be used for other processing, while still maintaining the advantages of step-level parallelism, such as... Figure 7 As shown. For example, suppose DFG 500 is modified to have two additional available cells when the addition cell becomes available. Without vector / matrix processing, when thread 810 marks the addition cell at 817, threads 830 and 840 will be occupied to mark the additional cells. In contrast, with thread 810 able to mark all available cells, threads 830 and 840 can begin evaluating the other two cells at the times corresponding to steps 836 and 845.

[0100] In partially ordered execution workflows, locking / unlocking mechanisms are also needed to prevent conflicts or contention between threads. In this case, locking strategies can be applied at the step level, rather than locking units throughout the entire execution flow. In some embodiments, the processor's running thread can lock the instruction unit at least according to the execution step when executing the execution step. In other words, the locking strategy is determined and applied to the execution step of the instruction unit.

[0101] The specific locks applied to an execution step depend on the nature of the step being implemented. For example, for an evaluation step, locking the cell being evaluated might be sufficient. For instance, a locking strategy for a deletion step could be to set a binary flag in the corresponding cell, marking it as "deleted." For steps that can be executed in parallel with another step, the locking strategy needs to account for that parallelism and support the cell being captured by a sufficient number of threads, but not more.

[0102] For example, a cell execution workflow may include an instantiation step. This can be done when the metadata of an instruction cell indicates that it needs to instantiate another cell. The responsible thread can then perform this step on the instruction cell to instantiate the required cell and update the metadata accordingly. During the execution of the instantiation step, an entire set of instruction cells can be locked so that the thread can find the cell to instantiate and populate it with data.

[0103] The above example of locking units at the granularity of execution steps is merely illustrative. According to the invention, locking for the same type of steps can vary in different implementations. Step-level locking improves thread safety by avoiding locking only for a single thread throughout the workflow.

[0104] Figure 9 A schematic block diagram of a device 900 that can be used to implement embodiments of the present invention is shown. Device 900 may be the device or apparatus described in the embodiments of the present invention, such as computing device 110. Figure 9 As shown, device 900 includes processor 901, which can be used to perform various appropriate actions and processes to perform the methods of the present invention (e.g., method 200). Processor 901 is implemented in hardware, firmware, software, or a combination of hardware and software. Furthermore, although... Figure 9 As not shown in the diagram, device 900 may also include a coprocessor.

[0105] Processor 901 can perform actions and processes according to computer program instructions to execute the method of the present invention. The computer program instructions used to perform the operations of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​and traditional procedural programming languages). In some embodiments, electronic circuits such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) are customized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of the present invention.

[0106] These computer-readable program instructions can be provided to the processing unit of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to form a machine, such that, when executed by the processing unit of the computer or other programmable data processing apparatus, the instructions produce modules for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Therefore, a computer-readable medium storing instructions includes an article of writing containing instructions that implement aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device, such that a series of operational steps may be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer, other programmable data processing apparatus or other device can perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0108] For example, computer program instructions can be stored in read-only memory (ROM) 902 or loaded from storage unit 908 into random access memory (RAM) 903. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904. The various methods or processes described above can be executed by processor 901.

[0109] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard and mouse; output unit 907, such as various types of monitors and speakers; storage unit 908, such as disk and optical disk; and communication unit 909, such as network card, modem, and wireless communication transceiver. Communication unit 909 enables device 900 to exchange information / data with other devices through computer networks (such as the Internet) and / or various telecommunications networks.

[0110] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for performing various aspects of the present invention.

[0111] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. For example, a computer-readable storage medium can be, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the above. Specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanical encoding devices (e.g., punched cards or raised structures in recesses on which instructions are recorded), and any suitable combination of the above. The computer-readable storage medium used in this document should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded to external computers or external storage devices via networks such as the Internet, local area networks, wide area networks, and / or wireless networks. Networks may include copper transmission cables, optical fiber transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives and forwards the computer-readable program instructions from the network for storage in a computer-readable storage medium within each computing / processing device.

[0113] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may appear in a different order than those marked in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and sometimes they may be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of special hardware and computer instructions.

[0114] Various embodiments of the invention have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the illustrated embodiments. The terminology used herein is chosen to best explain the principles and practical application of the various embodiments or to improve upon existing technology, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

1. A method for processing instruction units, characterized in that, The method includes: The first thread running on the processor executes the first execution step of a set of execution steps on the instruction unit within a first time period; During a second time period, a second thread running on the processor executes a second execution step of the set of execution steps on the instruction unit, wherein the second time period at least partially overlaps with the first time period.

2. The method according to claim 1, characterized in that, The method further includes: The instruction unit and its metadata are loaded into memory, wherein the metadata indicates one of a plurality of execution states.

3. The method according to claim 2, characterized in that, Executing the first execution step on the instruction unit includes: Based on the metadata of the instruction unit, the instruction unit is transitioned from a first execution state among a plurality of execution states to a second execution state among the plurality of execution states.

4. The method according to claim 3, characterized in that, Transitioning the instruction unit from the first execution state to the second execution state includes: Based on the execution of the first execution step, the metadata of the instruction unit is updated to indicate the second execution state.

5. The method according to claim 2, characterized in that, The method further includes: The first thread selects the instruction unit from a set of instruction units based on the metadata of the instruction unit, and executes the first execution step.

6. The method according to claim 5, characterized in that, Selecting the instruction unit includes: The metadata of the instruction unit is determined to meet the requirements for executing the first execution step.

7. The method according to claim 5, characterized in that, The metadata of the instruction unit indicates one or more inputs to the instruction unit for evaluation; Executing the first execution step on the instruction unit includes: In response to determining that the one or more inputs are available, the instruction unit is evaluated based on the one or more inputs.

8. The method according to claim 7, characterized in that, The evaluation of the instruction unit includes: The operation corresponding to the instruction unit is evaluated using one or more of the inputs; The output of the evaluation is stored in the metadata of the instruction unit.

9. The method according to claim 5, characterized in that, The metadata of the instruction unit indicates the output of the evaluation of the operation corresponding to the instruction unit; Executing the first execution step on the instruction unit includes: In response to determining that the output is available in the metadata, one or more instruction units are determined from the set of instruction units; The output is stored in the metadata of the one or more instruction units as input for evaluating the one or more instruction units.

10. The method according to claim 5, characterized in that, Each instruction unit in the set of instruction units corresponds to a node in the dataflow graph (DFG) of the program to be executed.

11. The method according to claim 1, characterized in that, The method further includes: During a third time period, a third thread running on the processor executes the third execution step of the set of execution steps on the instruction unit. During the execution of the third execution step, at least the instruction unit is locked according to the third execution step.

12. The method according to claim 1, characterized in that, The method further includes: The fourth thread running on the processor simultaneously executes the same type of execution steps on multiple instruction units.

13. A processor, characterized in that, The processor is used for: The first thread running on the processor executes the first execution step of a set of execution steps on the instruction unit during a first time period; The second execution step of the set of execution steps is executed on the instruction unit by a second thread running on the processor during a second time period, wherein the second time period at least partially overlaps with the first time period.

14. An electronic device, characterized in that, The electronic device includes: The processor according to claim 13; A memory coupled to the processor is used to store instruction units executed by the processor.

15. A computer program product, characterized in that, The method is tangibly stored on a computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 12.

Images