Method and system for inter-pipeline data hazard avoidance

By using counters to track and adjust the instruction execution order in the parallel processing unit, the problem of inter-pipe data hazards in the parallel processing unit is solved, and more efficient instruction execution and performance improvements are achieved.

CN119089854APending Publication Date: 2024-12-06IMAGINATION TECH LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411111950.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2017-12-07
Filing Date
2018-06-15
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In parallel processing units, it is difficult for the prior art to effectively avoid data hazards between pipelines, especially between multiple parallel instruction pipelines, resulting in complex data sequence execution and reduced performance.

Method used

By identifying data hazards between pipelines at compile time and using counters to track and adjust the order of instruction execution, we ensure that relevant instructions are executed only when the hazard is resolved, and the counter block and monitor logic are used to monitor and adjust the order of instruction execution to prevent data The occurrence of hazards.

Benefits of technology

It effectively avoids data hazards between pipelines, improves the accuracy and performance of instruction execution, reduces the cost of hardware implementation, and improves the overall performance of the processing unit.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089854A_ABST
    Figure CN119089854A_ABST
Patent Text Reader

Abstract

The invention relates to a method and system for inter-pipeline data hazard avoidance. Inter-pipeline data hazards are identified at a compilation time. For each identified inter-pipeline data hazard, where the primary instruction and the secondary instruction (s) are so identified and linked by a counter for tracking the inter-pipeline data hazard. Then, when the master instruction is output by the instruction decoder for execution, the value of the counter associated therewith is adjusted (e.g., increased) to indicate that there is a hazard associated with the master instruction, and when the master instruction has been resolved by the plurality of parallel processing pipelines, the value of the counter associated therewith is adjusted (e.g., increased) to indicate that there is a hazard associated with the master instruction. ) to indicate that the hazard associated with the master instruction has been resolved. When the decoder outputs a secondary instruction for execution, the secondary instruction is stagnated in a queue associated with an appropriate instruction pipeline if at least one counter associated with a primary instruction on which it depends indicates that there is a hazard associated with the primary instruction.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of Chinese invention patent application No. 201810621075.X, filed on June 15, 2018, and entitled “Method and system for avoiding data hazards between pipelines”. Technical Field

[0002] The present disclosure generally relates to methods and systems for inter-pipeline data hazard avoidance. Background Art

[0003] As known to those skilled in the art, data hazards are created in processing units having an instruction pipeline when the pipeline of instructions changes the order of read and write accesses to instruction operands such that the order is different from the order that would occur if the instructions were executed sequentially one by one.

[0004] There are three types of data hazards: read-after-write (RAW) hazards; read-after-write (WAR) hazards; and write-after-write (WAW) hazards, which are named based on the order in the program that must be preserved by the pipeline. RAW data hazards are the most common type of data hazard, and occur when a later instruction (relative to the order of instructions in the program) attempts to read a source operand before an earlier instruction has written to that source operand. This causes the later instruction to get the old value of the operand. For example, if there is a set of instructions like this:

[0005] R1=R2+R3

[0006] R4=R1-R5

[0007] Where a first instruction causes the sum of the values ​​of register 2 (R2) and register 3 (R3) to be stored in register 1 (R1), and a second instruction causes the difference between the value of register 1 (R1) and register 5 (R5) to be stored in register 4 (R4), a RAW data hazard occurs if the second instruction reads register 1 (R1) before the first instruction writes to register 1 (R1). A WAW data hazard occurs when a later instruction (relative to the order of instructions in the program) writes an operand before the operand is written by an earlier instruction, causing the writes to be performed in the wrong order so that the operand has the value from the earlier instruction instead of the value from the later instruction. A WAR data hazard occurs when a later instruction (relative to the order of instructions in the program) attempts to write an operand before the operand is read by an earlier instruction, causing the earlier instruction to read an incorrect value.

[0008] There are many known methods (such as forwarding) for avoiding data hazards caused by a single instruction pipeline, but many processing units such as graphics processing units (GPUs) are configured with multiple parallel instruction pipelines to effectively process large amounts of data in parallel. In such parallel processing units, it is necessary to track and eliminate not only intra-pipeline hazards (i.e., hazards associated with instructions executed in the same instruction pipeline), but also inter-pipeline hazards (i.e., hazards associated with instructions executed in different instruction pipelines).

[0009] The embodiments described below are provided by way of example only and are not intended to limit implementations that address any or all of the shortcomings of known GPUs or parallel processing units. Summary of the invention

[0010] This Summary is provided to introduce a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0011] Described herein are methods and parallel processing units for avoiding inter-pipeline data hazards, wherein inter-pipeline data hazards are identified at compile time. For each identified inter-pipeline data hazard, a primary instruction and (one or more) secondary instructions are identified and linked by counters for tracking inter-pipeline data hazards. When a primary instruction is output by an instruction decoder for execution, the value of the counter associated therewith is adjusted (e.g., increased) to indicate that there is a hazard associated with the primary instruction, and when the primary instruction has been resolved by multiple parallel processing pipelines, the value of the counter associated therewith is adjusted (e.g., decreased) to indicate that the hazard associated with the primary instruction has been resolved. When the decoder outputs a secondary instruction for execution, if at least one counter associated with the primary instruction on which it depends indicates that there is a hazard associated with the primary instruction, the secondary instruction is stalled in a queue associated with the appropriate instruction pipeline.

[0012] A first aspect provides a parallel processing unit, comprising: a plurality of counters; a plurality of queues, each queue preceding an instruction pipeline in a plurality of instruction pipelines; an instruction decoder configured to: decode a received instruction; in response to determining that the decoded instruction is a main instruction on which at least one other instruction depends, adjust the value of a counter in a plurality of counters associated with the main instruction to indicate that there is a hazard associated with the main instruction; and forward the decoded instruction to one of the plurality of queues; and a monitor logic configured to monitor the plurality of instruction pipelines, and in response to detecting that the instruction pipeline has resolved the hazard associated with the main instruction, adjust the value of the counter associated with the main instruction to indicate that the hazard associated with the main instruction has been resolved; wherein each queue is configured to, in response to receiving a secondary instruction that depends on one or more main instructions, stall the execution of the secondary instruction by the associated instruction pipeline if the counter associated with the main instruction on which the secondary instruction depends indicates that there is a hazard associated with the main instruction.

[0013] A second aspect provides a method for avoiding data hazards in a parallel processing unit, the method comprising: decoding an instruction by an instruction decoder; in response to determining at the instruction decoder that the decoded instruction is a main instruction on which at least one other instruction depends, causing the value of a counter in a plurality of counters associated with the main instruction to be adjusted to indicate that there is a hazard associated with the main instruction; and forwarding the decoded instruction from the instruction decoder to a queue in a plurality of queues, each queue receiving instructions to be executed by one of a plurality of instruction pipelines; in response to determining at the queue that the received instruction is a secondary instruction that depends on one or more main instructions, if the counter associated with the main instruction on which the secondary instruction depends indicates that there is a hazard associated with the main instruction, stalling the secondary instruction to be executed by the associated instruction pipeline; and in response to detecting by monitor hardware logic that the hazard associated with the main instruction has been resolved by an instruction pipeline in a plurality of instruction pipelines, causing the value of the counter associated with the main instruction to be adjusted to indicate that the hazard associated with the main instruction has been resolved.

[0014] A third aspect provides a computer-implemented method for generating computer executable instructions for a parallel processing unit, the method comprising: by a processor: receiving multiple related instructions; identifying data hazards in the multiple related instructions, each data hazard including a main instruction and one or more secondary instructions; assigning a counter of multiple counters to each main instruction to track the identified data hazards; generating a computer executable instruction for each main instruction, the computer executable instruction including information indicating that the computer executable instruction is a main instruction and information identifying the counter assigned to the main instruction; and generating a computer executable instruction for each secondary instruction, the secondary instruction including information identifying that the computer executable instruction is a secondary instruction and information identifying the counter assigned to the corresponding main instruction; and loading the computer executable instructions into the parallel processing unit.

[0015] The parallel processing unit described herein may be implemented in hardware on an integrated circuit. A method of manufacturing the parallel processing unit described herein in an integrated circuit manufacturing system may be provided. An integrated circuit definition data set may be provided that, when processed in the integrated circuit definition data set, configures the system to manufacture the parallel processing unit described herein. A non-transitory computer-readable storage medium having a computer-readable description of an integrated circuit stored therein may be provided that, when processed in the integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture the parallel processing unit described herein.

[0016] An integrated circuit manufacturing system can be provided, comprising: a non-transitory computer-readable storage medium having stored thereon a computer-readable integrated circuit description describing a parallel processing unit described herein; a layout processing system configured to process the integrated circuit description to generate a circuit layout description of an integrated circuit embodying the parallel processing unit described herein; and an integrated circuit generation system configured to manufacture the parallel processing unit described herein based on the circuit layout description.

[0017] A computer program code for performing the method as described herein may be provided.A non-transitory computer readable storage medium having computer readable instructions stored thereon may be provided, which when executed at a computer system causes the computer system to perform the method as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention is described by way of example with reference to the accompanying drawings. In the drawings:

[0019] Figure 1 is a block diagram of a first example parallel processing unit;

[0020] Figure 2 yes Figure 1 A schematic diagram of the format of the instruction;

[0021] Figure 3 yes Figure 1 A block diagram of an example queue of;

[0022] Figure 4 yes Figure 1 A block diagram of an example counter block;

[0023] Figure 5 is a flow chart of an example method for avoiding inter-pipeline data hazards;

[0024] Figure 6 is a block diagram of a second example parallel processing unit;

[0025] Figure 7 yes Figure 6 A block diagram of an example counter block;

[0026] Figure 8 is a flow chart of a second example method for avoiding inter-pipeline data hazards;

[0027] Fig. 9 is a block diagram of a third example parallel processing unit;

[0028] Fig.10 yes Fig. 9 Schematic diagram of an example counter block;

[0029] Fig.11 is a flow chart of a third example method for avoiding inter-pipeline data hazards;

[0030] Fig.12 is a flow chart of an example method of generating instructions for execution on a parallel processing unit;

[0031] Fig.13 is a flow chart of an example method for allocating counters to master instructions;

[0032] Fig.14 is a block diagram of an example computer system in which the parallel processing units described herein are implemented; and

[0033] Fig.15 is a block diagram of an example integrated circuit fabrication system for generating an integrated circuit embodying a system for implementing the parallel processing units described herein. DETAILED DESCRIPTION

[0034] The following description is given by way of example to enable those skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to those skilled in the art. The embodiments are described only by way of example.

[0035] As described above, many processing units such as GPUs include multiple parallel instruction pipelines that are designed to efficiently process large amounts of data in parallel. For example, some processing units may include a set of parallel instruction pipelines that include at least two instruction pipelines, each optimized for a specific type (or types) of calculations. Having multiple instruction pipelines configured to perform different types of calculations allows slow or rarely used instructions to be executed in parallel with high throughput common arithmetic operations, so that slow or rarely used instructions do not become a bottleneck. This also allows the arithmetic logic unit (ALU) of each pipeline to be optimized separately for its specific use.

[0036] While multiple instruction pipelines allow for more efficient use of processing resources (e.g., arithmetic logic unit (ALU) resources) and allow stalls caused by resource contention to be hidden, reordering instructions across multiple instruction pipelines complicates tracking data hazards to ensure instructions are executed in the correct order.

[0037] Specifically, in such a parallel processing unit, it is necessary to track and eliminate not only intra-pipeline hazards (i.e., hazards associated with instructions executed in the same instruction pipeline), but also inter-pipeline hazards (i.e., hazards associated with instructions executed in different instruction pipelines). Specifically, since there are multiple instruction pipelines running in parallel, related instructions may be in different pipelines (with different processing rates) at the same time. Therefore, a mechanism is needed to ensure that if there is a data hazard between instructions executed in different pipelines, the related instructions will not be executed before the data hazard is cleared.

[0038] Detecting inter-pipeline data hazards in hardware alone is very expensive in terms of area since a large number of pipeline stages would need to be tracked and a large number of comparisons would be required.

[0039] Thus, described herein are software controlled methods and systems for avoiding inter-pipeline data hazards in a GPU or other parallel processing unit having multiple parallel instruction pipelines (such as for high performance computing applications). Specifically, in the methods and systems described herein, inter-pipeline data hazards are identified at build time (e.g., by a compiler), and information is inserted into instructions identifying primary instructions (i.e., instructions on which one or more instructions in another pipeline depend) and secondary instructions (i.e., instructions that depend on one or more primary instructions in another pipeline), and the primary and secondary instructions are linked by a counter that is used to track inter-pipeline data hazards and enforce proper ordering of instructions.

[0040] When the instruction decoder of the parallel processing unit outputs a primary instruction for execution, the associated counter is modified (e.g., increased) to indicate that there is a hazard associated with the primary instruction (i.e., it is unsafe to execute the secondary instruction that depends on the primary instruction). When it is subsequently detected that the hazard associated with the primary instruction has been resolved (e.g., the primary instruction has written data to the memory), the value of the associated counter is adjusted (e.g., decreased) to indicate that the hazard associated with the primary instruction has been resolved (i.e., it is safe to execute the secondary instruction that depends on the primary instruction). The instructions for execution output by the instruction decoder are sent to a queue associated with the appropriate instruction pipeline. Before sending the instruction for execution from the queue to the instruction pipeline, the queue checks the (one or more) counters associated with the (one or more) primary instructions on which the secondary instruction depends for each secondary instruction. As long as at least one of the (one or more) counters associated with the (one or more) primary instructions on which the secondary instruction depends indicates that there is a hazard, the secondary instruction is stalled in the queue.

[0041] Stalling a secondary instruction just before being executed by an instruction pipeline shows enhanced performance when the primary instruction(s) upon which the secondary instruction depends will be completed quickly (e.g., when the primary instruction is executed by an instruction pipeline with high throughput). Such inter-pipeline data hazards may be referred to herein as low-latency inter-pipeline data hazards. However, stalling a secondary instruction just before being executed by an instruction pipeline shows reduced performance when the primary instruction(s) upon which the secondary instruction depends will be completed slowly (e.g., when the primary instruction is executed by an instruction pipeline with low throughput). Such inter-pipeline data hazards may be referred to herein as high-latency inter-pipeline data hazards.

[0042] Therefore, in some embodiments described herein, the compiler may be configured to separately identify and mark low-latency inter-pipeline data hazards and high-latency inter-pipeline data hazards. In these embodiments, the low-latency inter-pipeline data hazard may be processed as described above (e.g., when the instruction decoder outputs a primary instruction of the low-latency inter-pipeline data hazard, the value of the counter associated with the primary instruction is adjusted to indicate that there is a hazard associated with the primary instruction), and when it is subsequently detected that the hazard associated with the primary instruction has been resolved (e.g., the primary instruction has written data to the memory), the value of the counter associated with the primary instruction is adjusted to indicate that the hazard associated with the primary instruction has been resolved; and as long as the value of at least one counter associated with the primary instruction on which the secondary instruction associated with the low-latency data hazard output by the instruction decoder for execution depends indicates that there is a hazard, the secondary instruction is stalled in the queue before the appropriate instruction pipeline.

[0043] However, high latency inter-pipeline data hazards are handled differently. Specifically, a primary instruction with a high latency inter-pipeline data hazard is handled in the same manner as a primary instruction with a low latency inter-pipeline data hazard (e.g., when a primary instruction with a high latency inter-pipeline data hazard is output by a decoder for execution by an instruction pipeline, the value of a counter associated therewith is adjusted to indicate that there is a data hazard associated with the primary instruction, and when it is subsequently detected that the hazard has been resolved (e.g., the primary instruction has been written to memory), the value of a counter associated therewith is adjusted to indicate that the data hazard associated with the primary instruction has been resolved). However, when at least one secondary instruction with a high latency data hazard is decoded by an instruction decoder, a determination is made as to whether the associated high latency data hazard(s) have been resolved (i.e., whether the value of a counter associated with the dependent primary instruction(s) indicates that the high latency hazard has been resolved). If the associated high latency hazard has been resolved, the decoder outputs the secondary instruction for execution by the appropriate instruction pipeline. However, if the associated high latency hazard has not been resolved, the secondary instruction is descheduled by the instruction decoder (e.g., sent back to the scheduler) until the associated high latency hazard has been resolved (i.e., until the counter associated with the primary instruction on which it depends indicates that the hazard has been resolved). Once the associated high latency hazard of the secondary instruction has been resolved, the secondary instruction is rescheduled and sent back to the instruction decoder for processing.

[0044] In some cases, a secondary instruction may depend on both a high-latency primary instruction and a low-latency primary instruction. In these cases, the secondary instruction will be affected by the above-mentioned inter-pipeline hazard avoidance mechanism at the same time. Specifically, the instruction decoder will check the counter associated with the high-latency primary instruction, and the queue will be configured to check the counter associated with the low-latency primary instruction.

[0045] Although the methods, systems, and techniques described herein are described as being used for inter-pipeline data hazard avoidance, the methods, systems, and techniques described herein may also be used for intra-pipeline data hazards. For example, in cases where the tradeoff between area and performance does not justify the cost of having cycle-accurate hazard detection that can be achieved by other means, the methods, systems, and techniques described herein may also be used for intra-pipeline data hazard avoidance. In these cases, the compiler will be configured to also identify intra-pipeline data hazards and update its major and minor instructions in the same manner as described herein.

[0046] Reference now Figure 1 , Figure 1 A first example parallel processing unit 100 is shown which may be a GPU or other parallel processing unit. It should be understood that Figure 1 Only some elements of PPU 100 are shown, and there may be other components within the PPU that are not shown. Figure 1 Many other elements shown in (e.g., caches, interfaces, etc.). Figure 1 The PPU 100 includes a counter block 102 having a plurality of counters 103 , an instruction decoder 104 , a plurality of instruction pipelines 106 , a monitor logic 108 , and a queue 110 preceding each instruction pipeline 106 .

[0047] The counter block 102 includes a plurality of counters 103, which are used to track inter-pipeline data hazards and to order instructions according to them. Specifically, the counters 103 are used to indicate (i) when there is a hazard associated with a main instruction, and therefore it is unsafe for a secondary instruction that depends on the main instruction to be executed (e.g., (one or more) secondary instructions should be stalled); (ii) when the hazard associated with the main instruction is resolved, and therefore it is safe to execute the secondary instruction that depends on the main instruction. Specifically, the counter 103 is configured so that when the counter has a predetermined value or a set of values, it indicates that there is a hazard associated with the associated main instruction; and when the counter has a different predetermined value or a set of predetermined values, it indicates that the hazard associated with the associated main instruction has been resolved. In some examples, the counter 103 is configured so that when the counter has a non-zero value, it indicates that there is a hazard associated with the associated main instruction, and when the counter has a zero value, it indicates that the hazard associated with the associated main instruction has been resolved. It is obvious to those skilled in the art that this is only an example, and the counter 103 can be configured so that different values ​​indicate that a hazard exists and / or that the hazard has been resolved.

[0048] When the counter 103 indicates that there is a hazard associated with the associated primary instruction, the counter 103 acts as a guardrail that the secondary instructions on which it depends have reached. Specifically, the secondary instructions must wait until the guardrail is removed. Accordingly, Figure 1 The counter 103 may be referred to herein as a guardrail counter.

[0049] The counter block 102 is configured to adjust the value of the counter 103 in response to receiving an adjustment instruction or signal from the instruction decoder 104, the monitor logic 108, and the optional queue 110; and generate counter status information and provide it to the queue 110. Specifically, as described in more detail below, the instruction decoder 104 is configured to send an adjustment instruction or signal to the counter block 102 in response to outputting a host instruction (e.g., in response to forwarding the host instruction to the queue 110) for execution, which causes the counter block 102 to adjust the value of the counter 103 associated with the host instruction to indicate that the host instruction is compromised. The monitor logic 108 is configured to send an adjustment instruction or signal to the counter block 102 in response to detecting that the hazard associated with the host instruction has been (partially or completely) resolved by the instruction pipeline 106, which causes the counter block 102 to adjust the value of the counter 103 associated with the host instruction to indicate that the hazard associated with the host instruction has been (partially or completely) resolved. The queue 110 may also be configured to, in response to detecting that a primary instruction is no longer (partially or completely) valid and therefore the primary instruction is (partially or completely) discarded, send an adjustment instruction or signal to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the primary instruction to indicate that the hazard associated with the primary instruction has been (partially or completely) resolved.

[0050] The counter status information includes information indicating whether there is a hazard associated with the main instruction associated with each counter or whether the hazard has been cleared or has been resolved. The counter block 102 is configured to generate the counter status information based on the value of the counter 103. In some cases, the counter status information may include a flag or bit for each counter 103, which indicates whether there is a hazard associated with the corresponding main instruction or whether the hazard associated with the corresponding main instruction has been resolved. For example, the counter status information may include a single bit flag for each counter 103, where the flag is set to "1" to indicate that there is a hazard associated with the main instruction, and the flag is set to "0" to indicate that the hazard associated with the main instruction has been resolved. In other cases, the counter status information may include the value of each counter 103, and the recipient of the counter status information is configured to determine whether the hazard associated with the associated main instruction has been resolved based on these values.

[0051] The instruction decoder 104 receives an instruction that includes information identifying a primary instruction (i.e., an instruction that at least one other instruction in another instruction pipeline depends on), a secondary instruction (i.e., an instruction that depends on at least one primary instruction in another pipeline), and their associated counters (e.g., inserted by a compiler at build time). Specifically, each primary instruction will be assigned a counter, and (one or more) secondary instructions will be linked to the primary instruction through the counter. Since a secondary instruction may depend on more than one primary instruction, a secondary instruction may be linked to multiple primary instructions through multiple counters. Figure 2 Examples of information and information formats that identify major and minor instructions and their associated counters are described. Typically, there are fewer counters than inter-pipeline data hazards, so counters are often reused for multiple inter-pipeline data hazards.

[0052] The instruction decoder 104 decodes the received instructions, selects the appropriate instruction pipeline for executing each instruction, and outputs the instruction for execution by the selected instruction pipeline. If the instruction decoder 104 determines that the instruction output for execution is a primary instruction, the instruction decoder sends an adjustment instruction or signal to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the primary instruction to indicate that there is a hazard associated with the primary instruction (therefore, it is unsafe to execute the secondary instructions that depend on the primary instruction). For example, if the primary instruction X is associated with counter 2, then when the instruction decoder 104 outputs the primary instruction X for execution, the instruction decoder 104 will output an adjustment signal or instruction to the counter block 102, causing the counter block 102 to adjust the value of counter 2 to indicate that there is a hazard associated with the primary instruction X.

[0053] In some examples, the instruction decoder 104 can be configured to output an adjustment instruction or signal in response to outputting the main instruction for execution, which adjustment instruction or signal causes the counter block 102 to increase the counter 103 associated with the main instruction by a predetermined amount (e.g., 8). In some cases, as described below, each instruction can be part of or associated with a task that causes multiple instances of the instruction to be executed. In these cases, it is safe to execute the secondary instruction only when the hazards for all instances of the main instruction have been resolved. In this case, the predetermined amount by which the counter 103 is increased can reflect the number of instances or groups of instances that can track the hazards separately. For example, if 32 instances of the instruction can be executed in groups of 4 (i.e., 8 groups), the instruction decoder 104 can be configured to increase the counter 103 by 8.

[0054] Each instruction pipeline 106 includes hardware logic (e.g., one or more ALUs) for executing instructions. In some examples, the multiple instruction pipelines 106 include at least two different instruction pipelines configured to execute different types of decoded instructions. For example, the instruction pipeline 106 may include one or more instruction pipelines configured to: (i) perform bit integer operations, floating point operations, and logical (bitwise) operations; (ii) calculate texture coordinates or other changes per instance; (iii) perform 32-bit floating point non-rational / transcendental operations; (iv) perform 64-bit floating point operations; (v) perform data copying and format conversion; (vi) perform texture address calculations; and (vii) perform atomic operations on local memory registers. Having multiple instruction pipelines configured to execute different types of instructions allows slow or rarely used instructions to be executed in parallel with high-throughput common arithmetic operations so that slow or rarely used instructions do not become a bottleneck. This also allows the ALUs to be optimized for their specific uses respectively.

[0055] In some cases, instruction pipeline 106 can each be a single instruction multiple data (SIMD) pipeline. As known to those skilled in the art, a SIMD instruction is an instruction that makes the same operation performed on multiple data items associated with the instruction when executed. SIMD instructions allow fewer instructions to specify the same workload, thereby reducing the pressure on instruction fetch modules and instruction decoders. Therefore, a SIMD pipeline is a pipeline that can handle SIMD instructions, that is, it is a pipeline that can execute the same instruction on multiple data items. This means that if an instruction is a part of a task, as described in more detail below, an instruction pipeline 106 can use an issued instruction to perform an instance or data item of the entire task. The instruction pipeline may need to process the issued SIMD instruction more than one clock cycle.

[0056] The monitor logic 108 monitors the instruction pipeline 106 to detect when the hazard associated with the main instruction has been resolved (partially or completely) by the instruction pipeline, and sends an adjustment instruction or signal to the counter block 102 in response to detecting that the hazard associated with the main instruction has been resolved (partially or completely) by the instruction pipeline 106, so that the counter associated with the main instruction indicates that the hazard associated with the main instruction has been resolved (partially or completely). For example, if the main instruction is associated with Counter 2, then when the monitor logic 108 detects that the instruction pipeline 106 has resolved the hazard associated with the main instruction, the monitor logic 108 will send an adjustment signal or instruction to the counter block 102 so that the counter block 102 adjusts the value of Counter 2 to indicate that the hazard associated with the main instruction has been resolved.

[0057] In some examples, monitor logic 108 can be configured to, in response to detecting that a hazard associated with a host instruction has been (partially or fully) resolved by instruction pipeline 106, send an adjustment signal or instruction to counter block 102, causing counter block 102 to reduce the value of counter 103 by a predetermined amount (e.g., 1 or 8) to indicate that the hazard has been (partially or fully) resolved.

[0058] As described in more detail below, in some cases, each instruction may be part of or associated with a task that results in multiple instances of the instruction being executed (e.g., up to 32 instances). In these cases, when all instances have resolved the hazard, the hazard is said to be fully resolved, and when some (but not all) instances have resolved the hazard, the hazard is said to be partially resolved. The instances may be divided into multiple groups (e.g., 8), and each group is executed as a block, so that the execution of each block can be tracked separately. In these cases, the monitor logic 108 may be configured to send a separate instruction or signal each time it is detected that a hazard associated with a primary instruction has been resolved by a group of instances, so that the value of the counter is adjusted to indicate that the hazard has been partially resolved (e.g., the instruction decrements the value of the counter by 1). Once the hazard is resolved by each group, the counter will indicate that the hazard has been fully resolved. It will be apparent to those skilled in the art that this is merely an example, and the monitor logic 108 may be configured to cause the counter block 102 to adjust the value of the counter associated with the primary instruction in any suitable manner so that the counter will have a value indicating that the hazard associated with it has been resolved (fully or partially).

[0059] The monitor logic 108 may be configured to use different criteria to determine when a hazard has been resolved by the instruction pipeline 106 based on the type of hazard. For example, a WAW or RAW hazard may be resolved when a host instruction has written the result of the instruction to a storage device 112, such as a memory 114 or a register (not shown). Thus, the monitor logic 108 may be configured to detect that a WAW or RAW hazard has been resolved by the instruction pipeline 106 when the monitor logic 108 detects that the instruction pipeline 106 has written the result of the host instruction to the storage device 112. In these cases where each instruction pipeline 106 has an interface to a storage unit, the monitor logic 106 may be configured to monitor these instruction pipelines 106 to the storage interface to detect writes to the storage device. Conversely, a WAR hazard may be resolved when the instruction pipeline 106 has read the source of the host instruction. Thus, the monitor logic 108 may be configured to detect whether a WAR hazard has been resolved by the instruction pipeline 106 when the monitor logic 108 detects that the source of the host instruction has been read by the instruction pipeline 106.

[0060] Although the monitor logic 108 is Figure 1106 as a single logic block separate from the instruction pipeline, in other examples, the monitor logic 108 may be distributed in and be part of the instruction pipeline 106. For example, each instruction pipeline 106 may include its own monitor logic.

[0061] Each instruction pipeline 106 is preceded by a queue 110 that receives instructions from the instruction decoder 104 to be executed by the corresponding instruction pipeline 106 and forwards the received instructions to the corresponding instruction pipeline 106 for sequential execution. Each queue 110 is configured to determine whether the instruction is a secondary instruction before forwarding the instruction to the corresponding instruction pipeline 106 for execution. If the instruction is not a secondary instruction, the instruction is forwarded to the corresponding instruction pipeline 106 for execution. However, if the instruction is a secondary instruction, a determination is made (based on the counter 103 and / or counter status information) whether the hazard associated with the primary instruction on which the secondary instruction depends has been resolved. If the hazard associated with the primary instruction on which the secondary instruction depends has been resolved, the secondary instruction is forwarded to the corresponding instruction pipeline 106 for execution. However, if at least one hazard associated with the primary instruction on which the secondary instruction depends has not been resolved, the instruction is stalled.

[0062] Thus, instructions may be forwarded to the instruction pipeline 106 for execution only if all counters associated with the primary instruction(s) indicate that the associated hazards have been resolved. For example, if the queue 110 receives a secondary instruction that is dependent on a primary instruction associated with counters 2 and 3, the queue 110 may not forward the secondary instruction to the instruction pipeline until both counters 2 and 3 have values ​​(e.g., zero) indicating that the associated hazards have been resolved.

[0063] In some examples, stalling a secondary instruction may stall the execution of all subsequent instructions by the associated instruction pipeline. Figure 3 In more detail, in other examples where instructions are part of or associated with a task, when queue 110 stalls secondary instructions associated with a first task, it may be able to forward other subsequent instructions associated with a different task to an associated instruction pipeline.

[0064] In some cases, queue 110 may be configured to determine the value of the appropriate counter(s) by polling or requesting counter status information of the appropriate counters from counter block 102. In other cases, counter block 102 may be configured to push counter status information to queue 110 periodically.

[0065] In some cases, the queue 110 may also be configured to send an adjustment instruction or signal to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the main instruction to indicate that the hazard has been (partially or completely) resolved if the queue 110 detects that the instruction will be (partially or completely) discarded before forwarding. Specifically, it is possible that an instruction is sent to the queue 110 for execution due to various reasons, but when the instruction is issued to the instruction pipeline, it may no longer be desired that the instruction be executed.

[0066] For example, this may occur when a parallel processing unit implements prediction. As known to those skilled in the art, prediction is a process implemented in a parallel processing unit, which is an alternative to branch prediction. In branch prediction, a parallel processing unit predicts the branch path to be executed and predictively executes instructions related to the branch. An incorrect prediction (i.e., an incorrect guess as to which path of the branch will be taken) may cause the pipeline to stagnate and cause instructions to be taken from the actual branch target address. In contrast, in prediction, instructions related to all possible paths of the branch are executed in parallel, and only those instructions related to the adopted path (as determined by the branch condition) are allowed to modify the architectural state. Each instruction from a particular path will be associated with a predicate (e.g., a Boolean value) indicating whether the instruction is allowed to modify the architectural state. The predicate value will be set based on the evaluation of the branch condition. Instructions whose predicate indicates that the instruction is not allowed to modify the architectural state are said to have been predicted. If an instruction has been predicted before being forwarded to the instruction pipeline, it does not need to be forwarded to the instruction pipeline for execution.

[0067] Therefore, before forwarding the instruction to the instruction pipeline for execution, the queue 110 can be configured to determine whether it is desired to forward the instruction to the pipeline for execution based on active information (e.g., predicate information). If the active information indicates that the instruction is not to be executed (e.g., the predicate indicates that the instruction has been predicted), the instruction is discarded, and if the instruction is a secondary instruction, the queue 110 sends an adjustment instruction or signal to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the primary instruction to indicate that the hazard has been resolved (e.g., an instruction that causes the counter block to reduce the value of the counter by a predetermined amount).

[0068] In the case where the instruction is associated with a task, the instruction may be valid for some instances but not other instances. For example, if the instruction is predicted on a per-instance basis, this may occur. In these cases, the queue 110 can be configured to detect whether the instruction is partially valid (some but not all instances are valid), completely invalid (all instances are invalid), or completely valid (all instances are valid). If the queue 110 detects that the instruction is completely invalid, the queue 110 can send an adjustment instruction to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the main instruction to indicate that the hazard has been completely resolved (for example, an instruction that causes the counter block to reduce the value of the counter by 8), and if the queue 110 detects that the instruction is partially invalid, the queue 110 can send an adjustment instruction to the counter block 102, which causes the counter block 102 to adjust the value of the counter 103 associated with the main instruction to indicate that the hazard has been partially resolved (for example, an instruction that causes the counter block to reduce the value of the counter by less than 8 to reflect which part of the hazard of the instruction has been resolved).

[0069] The following references Figure 3 describe Figure 1 An example implementation of queue 110 is provided.

[0070] In some examples, the instructions decoded by the instruction decoder 104 and executed by the instruction pipeline 106 are all related to or correspond to tasks. The term "task" is used herein to refer to a work group or work body (i.e., multiple work items or program instances) in which the same program (e.g., the same ALU instruction sequence) is applied to a set of data, wherein the data may include one or more elements (or data items, such as multiple pixels or vertices). Therefore, a task involves one or more data items (one or more references / points to the data items) and a program to be executed on the data item (or a reference (e.g., a pointer) to the program (i.e., an instruction sequence). As described above, when executing instructions related to a task, an instance of the instruction is executed independently for each data item.

[0071] In these examples, tasks may be received at a scheduler 116, which schedules the received tasks for execution (i.e., determines the order in which the tasks will be executed). Once the scheduler 116 determines which task to execute next, the scheduler 116 sends information identifying the task (e.g., a task ID) and information identifying the next instruction to be fetched (e.g., a program counter value) to the instruction fetch module 120. The instruction fetch module 120 then fetches the next instruction for the task from memory (e.g., via a cache structure) based on the received information (e.g., the program counter value). The fetched instruction is then sent to the instruction decoder 104 to be decoded. Tasks that have been sent to the instruction fetch module 120 for execution are considered active tasks or scheduled tasks. The instruction fetch module 120 and the instruction decoder 104 may form part of the instruction controller 118.

[0072] Since tasks contain related instructions (e.g., instructions of the same program), there are usually dependencies (e.g., hazards) between instructions within a task, but there are usually no dependencies (e.g., hazards) between instructions in different tasks. Therefore, there may be multiple counters 103 associated with each task ID to track pipeline-to-pipeline hazards within a task by the task ID. In the following, Figure 4 An example of a counter block 102 is described in which a counter 103 is associated with one of the task IDs. In the case where the counters associated with each task ID are assigned the same counter ID (e.g., counter number), both the task ID and the counter ID can be used to identify or index a specific counter associated with an instruction.

[0073] Reference Figure 2 , which shows a diagram including information identifying major and minor instructions and counters associated therewith. Figure 1 An example format of instruction 200 in a system. Figure 2 In the example of , the instruction includes a primary instruction field 202 and a secondary instruction field 204. The primary instruction field 202 indicates whether the instruction is a primary instruction, and if so, which counter the primary instruction is associated with. Figure 2 In the example of , the primary instruction field 202 is configured to store an N-bit number. When the N-bit number has a specific value (e.g., zero), it indicates that the instruction is not a primary instruction, and when the N-bit number has a value different from the specific value (e.g., a non-zero value), it indicates that the instruction is a primary instruction and it identifies (e.g., by number) a counter associated with the primary instruction.

[0074] For example, a three-bit primary instruction field 202 having a binary value of “000” (i.e., a decimal value of zero) may indicate that the instruction is not a primary instruction (i.e., no other instructions in another instruction pipeline are dependent on the instruction), and a three-bit primary instruction field 202 having a binary value of “001” (i.e., a decimal value of 1) may indicate that the instruction is a primary instruction (i.e., there is at least one other instruction in another instruction pipeline that is dependent on the instruction) and that the primary instruction is associated with counter number 1.

[0075] The number of bits allocated to the main instruction field 202 (i.e., N) will be based on the number of counters 103. Specifically, the main instruction field 202 generally includes enough bits to uniquely identify each counter. For example, if there are 6 counters, the main instruction field may include 3 bits (which can represent 8 unique numbers) so that the six counters can be uniquely identified by unique numerical values.

[0076] The secondary instruction field 204 indicates whether the instruction is a secondary instruction and, if so, which counters are associated with the primary instruction on which it depends. Figure 2 In the example of , the secondary instruction field 204 is configured to hold an M-bit mask. Each counter has one bit, and the value of the bit indicates whether the instruction is dependent on the primary instruction associated with the counter. For example, if there are 6 counters, there will be 6 bits in the mask, the first bit can correspond to the first counter, the second bit can correspond to the second counter, the third bit can correspond to the third counter, and so on. In some cases, a set bit (e.g., "1") can indicate that the instruction is dependent on the primary instruction associated with the corresponding counter, and a cleared or unset bit (e.g., "0") can indicate that the instruction is not dependent on the primary instruction associated with the corresponding counter. In these cases, if at least one bit in the mask of the secondary instruction field 204 is set, the instruction is dependent on at least one instruction in another pipeline, and therefore the instruction is a secondary instruction; and if no bit in the mask of the secondary instruction field 204 is set, the instruction is not dependent on any instruction in another pipeline, and therefore the instruction is not a secondary instruction.

[0077] For example, a 6-bit sub-instruction field 204 having a binary value of “000000” may indicate that the instruction is not a sub-instruction (i.e., it is not dependent on any other instruction), and a 6-bit sub-instruction field 204 having a binary value of “010100” may indicate that the instruction is a sub-instruction that is dependent on instructions associated with the second and fourth counters.

[0078] Since an instruction can both depend on and be depended on by one or more other instructions, an instruction can be both a primary instruction and a secondary instruction.

[0079] As mentioned above, instructions (e.g., Figure 2The instruction 200 of the data hazard may be generated at build time by a compiler configured to identify inter-pipeline data hazards and to assign counters to each identified inter-pipeline data hazard for tracking inter-pipeline data hazards and enforcing appropriate ordering. The primary instruction of the data hazard is then modified to include information indicating that it is a primary instruction and a counter that has been associated with the primary instruction (e.g., in the primary instruction field 202); and the secondary instruction(s) of the data hazard are modified to include information indicating the counter associated with the primary instruction on which it depends (e.g., in the secondary instruction field 204).

[0080] In some cases, the compiler may be configured to allocate a counter for each identified inter-pipeline data hazard. In other cases, the compiler may be configured to use a single counter for multiple identified inter-pipeline data hazards. For example, if instructions C and D are to be executed in the same instruction pipeline and C depends on instruction A and instruction D depends on instruction B, then both main instructions A and B may be assigned the same counter. C will have to wait for the hazards associated with instructions A and B to be resolved before it can be executed, and since D follows C, D will also have to wait for the hazards associated with instructions A and B to be resolved. If instructions C and D are to be executed on different instruction pipelines, then they will both wait for the hazards associated with instructions A and B to be resolved, but they may wait too long. In the case where the secondary instruction depends on two different main instructions in different pipelines, it is preferred to use a single counter to handle multiple inter-pipeline data hazards. In this case, the main instructions can all be assigned to the same counter. Therefore, in some cases, there may be a performance cost in using a single counter for multiple inter-pipeline data hazards, but this can allow the use of a smaller number of counters, thereby reducing the size of the hardware implementation.

[0081] Reference now Figure 3 , showing the parallel processing unit Figure 1An example implementation of a queue 110 of , wherein each instruction corresponds to a task to which a task ID has been assigned, and each instruction pipeline 106 receives instructions associated with a plurality of tasks, each of which has a corresponding different task ID. As described above, the queue 110 receives instructions from the instruction decoder 104 to be executed by the corresponding instruction pipeline 106, and forwards the instructions to the corresponding instruction pipeline 106 in the correct order. However, no secondary instruction can be forwarded to the instruction pipeline 106 until the counter associated with the main instruction on which the secondary instruction depends indicates that the hazard associated with the main instruction has been resolved. In this example, the queue 110 preferably issues all instructions to the corresponding instruction pipeline in the order in which they are received, but the queue 110 must maintain the order of the instructions within the task. This means that if a secondary instruction is stalled because the counter associated with the (one or more) main instructions on which the secondary instruction depends does not indicate that the hazard has been resolved, only subsequent instructions associated with the same task (e.g., identified by the same task ID) also need to be stalled, and subsequent instructions associated with different tasks (e.g., identified by different task IDs) can be forwarded to the corresponding instruction pipeline 106 before the stalled secondary instruction.

[0082] exist Figure 3 In the example of , the queue 110 includes a first-in-first-out (FIFO) queue 302 , a hazard detection logic 304 , a hazard instruction queue 306 , an information queue 308 , and a multiplexer (MUX) 310 .

[0083] FIFO queue 302 receives instructions output by instruction decoder 104, stores the instructions in the order in which they are received, and then outputs them to hazard detection logic 304 in the same order (i.e., the order in which they are received). In some cases (e.g., when there is one instruction decoder), FIFO queue 302 may be able to receive at most one instruction per clock cycle. In other cases (e.g., when there are multiple instruction decoders), FIFO queue 302 may be able to receive more than one instruction per clock cycle. FIFO queue 302 generally can only output at most one instruction per clock cycle.

[0084] The hazard detection logic 304 is configured to determine whether the instruction output from the FIFO queue 302 is non-hazardous (i.e., whether the instruction is safe to execute). In this example, there are two potential hazards: (i) the instruction may be a secondary instruction that waits for (one or more) hazards associated with one or more primary instructions to be resolved before it can be issued to the instruction pipeline for execution; and / or (ii) the instruction may be related to the same task (e.g., has the same task ID) as the instructions in the hazard instruction queue 306, and therefore must wait for the instruction to be issued to the corresponding instruction pipeline before it can be issued to the instruction pipeline for execution. Since newer instructions associated with a task cannot be issued to the instruction pipeline before older instructions associated with the same task, if there are already instructions associated with the same task in the hazard instruction queue 306 that have not yet been issued to the instruction pipeline, the new instruction must wait for the older instructions to be issued to the instruction pipeline before it can be issued to the instruction pipeline.

[0085] In some cases, the hazard detection logic 304 is configured to first determine whether there are any instructions related to the same task (e.g., based on the task ID associated with the instruction) in the hazard instruction queue 306. If it is determined that there is at least one stalled instruction in the hazard instruction queue 306 with the same task ID, then the instruction cannot be safely executed and is sent to the hazard instruction queue 306 where it waits for an earlier instruction with the same task ID to be sent to the corresponding instruction pipeline 106.

[0086] However, if no instruction with the same task ID exists in the hazard instruction queue 306, then a determination is made as to whether the instruction received from the FIFO queue 302 is a secondary instruction (indicating a potential hazard). If the hazard detection logic 304 determines that the instruction received from the FIFO queue 302 is not a secondary instruction, then the instruction is a non-hazardous instruction (and is therefore ready to be executed), and the instruction is forwarded directly to the MUX 310. However, if the hazard detection logic 304 determines that the instruction received from the FIFO queue 302 is a secondary instruction (e.g., the secondary instruction field of the instruction is non-zero), then the hazard detection logic 304 determines whether the counter associated with the primary instruction upon which it depends indicates that the hazard associated with the associated primary instruction has been resolved (e.g., via counter status information).

[0087] If all dependent counters indicate that the hazard associated with the associated host instruction has been resolved (e.g., all dependent counters are zero), then the instruction is a non-hazardous instruction (and therefore ready to execute), and the instruction is forwarded directly to MUX 310. However, if at least one dependent counter indicates that the hazard associated with the associated host instruction has not been resolved (e.g., at least one dependent counter has a non-zero value), then the instruction is a hazardous instruction and is forwarded to hazardous instruction queue 306.

[0088] The hazard instruction queue 306 is configured to store hazard instructions in order until the corresponding hazard(s) have been cleared. The information queue 308 stores information for each hazard instruction in the hazard instruction queue 306, the information indicating the hazard(s) associated with the instruction. For example, the information queue 308 may store, for each hazard instruction, information indicating whether the hazard associated with each of the primary instructions on which it depends has been resolved and / or information indicating whether the instruction is waiting for another instruction in the hazard queue (and if so, which one).

[0089] As described above, there are two potential hazards in this example: (i) the instruction may be a secondary instruction that is waiting for a hazard associated with one or more primary instructions to be resolved; and / or (ii) the instruction may involve the same task ID as an instruction in the hazard instruction queue 306 and is waiting for the instruction to be issued to the corresponding instruction pipeline. Therefore, the information queue 308 may store a single bit or flag indicating whether the corresponding hazard instruction is waiting for the hazard associated with the primary instruction to be resolved (which may be set if at least one counter associated with the primary instruction on which the instruction depends indicates that the hazard has not been resolved, and may be cleared once all counters associated with the primary instructions on which the instruction depends indicate that the hazard has been resolved); and / or the information queue 308 may store a flag or bit indicating whether the instruction is waiting for another instruction in the hazard instruction queue 306 to be issued to the corresponding instruction pipeline.

[0090] When the information in the information queue 308 indicates that all hazards have been cleared for a particular instruction (for example, the counter value of the relevant counter indicates that the hazards associated with the main instruction on which it depends have been resolved and / or the waiting hazard instruction has been forwarded to the corresponding instruction pipeline), the hazard instruction queue 306 forwards the cleared hazard instruction to the MUX 310.

[0091] When the corresponding instruction pipeline 106 is ready to receive the next instruction, the MUX 310 is configured to select one of the instruction output by the hazard detection logic 304 and the instruction output by the hazard instruction queue 306 to be forwarded to the corresponding instruction pipeline 106 for execution. If only one of the hazard detection logic 304 and the hazard instruction queue 306 has output an instruction in the same clock cycle, the MUX 310 selects the instruction and forwards it to the corresponding instruction pipeline. However, if both the hazard detection logic 304 and the hazard instruction queue 306 output instructions in the same clock cycle, the MUX 310 selects the instruction output from the hazard instruction queue 306 because it is generally older than the instruction output by the hazard detection logic 304.

[0092] In some cases, the queue 110 may also include valid logic 312, which is configured to receive the instruction output by the MUX 310 and receive valid information indicating whether all or part of the instruction is valid. As described above, the instruction may be sent to the queue for execution, but when the instruction is sent to the instruction pipeline, it may no longer be required to be executed (for example, it may have been predicted). Therefore, before the instruction output by the MUX 310 is forwarded to the instruction pipeline for execution, the valid logic 312 is configured to determine whether it is desired to forward the instruction to the pipeline for execution based on the valid information. If the valid information indicates that the instruction is not executed, the instruction is discarded, and if the instruction is a main instruction, the active logic 312 sends an instruction to the counter block 102, which causes the counter block 102 to adjust the counter 103 associated with the main instruction to indicate that the hazard has been cleared (for example, an instruction that causes the counter block to reduce the value of the counter by a predetermined amount).

[0093] In the case where the instruction is associated with a task, the instruction may be valid for some instances but not others. In the case where the instances are divided into groups that can be tracked separately, the validity logic 312 can be configured to determine which groups of instances are valid (i.e., have at least one valid instance). If the instruction does not have a valid group, the instruction is completely invalid and the instruction is discarded. If the completely invalid instruction is a main instruction, then because the hazard will not be resolved or cleared by the instruction pipeline 106, the validity logic 312 sends an instruction to the counter block 102 so that the counter block 102 adjusts the value of the counter 103 associated with the main instruction to indicate that the hazard has been resolved (e.g., an instruction that causes the counter block to reduce the value of the counter by a predetermined amount).

[0094] However, if there is at least one valid group for the instruction, the instruction is partially invalid (or partially valid), and the valid logic 312 forwards the instruction to the instruction pipeline 106 for execution. If the partially invalid instruction is a host instruction, since instances of the invalid groups will not be executed, the hazard will not be resolved or cleared by the instruction pipeline 106, and the valid logic 312 sends an instruction to the counter block 102, which causes the counter block 102 to adjust the value of the counter to indicate that the hazard has been partially resolved (e.g., causes the counter block to reduce the value of the counter by the number of inactive groups).

[0095] Reference now Figure 4 , which shows Figure 1 An example implementation of the counter block 102 is shown, wherein each task ID is associated with a plurality of counters 103 . Figure 4 The counter block 102 includes a matrix 402 of counters 103 , counter update logic 404 , and state logic 406 .

[0096] As described above, instructions associated with or forming part of a task are dependent instructions, and therefore inter-pipeline data hazards may occur between instructions within a task. However, since tasks themselves are typically independent of other tasks, inter-pipeline data hazards may not occur between instructions of different tasks. Therefore, inter-pipeline data hazards can typically be identified and tracked on a per-task basis.

[0097] The matrix 402 of the counter 103 includes a row for each possible task ID. For example, in the case where there are 3 possible task IDs, the matrix 402 will have three rows, such as Figure 4 Each row includes K counters associated with a corresponding task ID, where K is an integer greater than or equal to 2. Therefore, the counter 103 in a particular row is used to track inter-pipeline data hazards of the task with the corresponding task ID. For example, in Figure 4 , the counters in the first row of matrix 402 are used to track inter-pipeline data hazards for tasks having task ID "01" (binary), and the second row of matrix 402 is used to track inter-pipeline data hazards for tasks having task ID "10" (binary).

[0098] As described above, at build time, inter-pipeline data hazards in a task are identified, and a primary instruction is modified to include information identifying the instruction as a primary instruction and information identifying a counter for tracking the hazard (e.g., a counter number); and a secondary instruction is modified to include information identifying the instruction as a secondary instruction and information identifying one or more counters for tracking the primary instruction(s) on which it depends. Thus, in addition to identifying inter-pipeline data hazards in a task and identifying its primary and secondary instructions, the compiler is configured to assign a counter to each identified inter-pipeline data hazard, which is used to track the corresponding data hazard. The compiler may be configured to assign counters (e.g., counter numbers) to the identified inter-pipeline data hazards in any suitable manner. In the case where there are a small number of available counters per task, in order to make it possible for a counter to be used multiple times for the same task, the compiler may be configured to assign counters to the identified inter-pipeline data hazards in a manner that maximizes the time between uses of the counter (e.g., the compiler may be configured to assign counters to the identified inter-pipeline data hazards on a cycle basis).

[0099] The counter update logic 404 receives an adjustment instruction or signal from the instruction decoder 104, the monitor logic 108, and optionally, the queue 110, which causes the value of the counter to be adjusted to reflect the current state of the corresponding inter-pipeline data hazard. Specifically, as described above, when the instruction decoder outputs a host instruction for execution, the instruction decoder 104 sends an adjustment instruction to the counter block 102 so that the counter 103 associated with the host instruction has a value indicating that there is a hazard associated with the host instruction. In some examples, the instruction decoder 104 is configured to send an adjustment instruction to the counter block 102, which causes the value of the counter 103 associated with the host instruction to increase by a predetermined amount (e.g., 8).

[0100] When the monitor logic 108 subsequently detects that the hazard associated with the subject instruction has been resolved (completely or partially) by one of the instruction pipelines (e.g., by monitoring the interface between the instruction pipeline and the storage module), the monitor logic 108 sends an adjustment instruction or signal to the counter block 102 so that the counter 103 associated with the subject instruction indicates that the hazard associated with the subject instruction has been resolved (partially or completely). In some examples, the monitor logic 108 is configured to send an adjustment instruction to the counter block 102 so that the value of the counter 103 associated with the subject instruction is reduced by a predetermined amount (e.g., 8).

[0101] If the queue 110 detects that the main instruction to be forwarded to the corresponding pipeline is invalid (in whole or in part), the queue 110 sends an adjustment instruction or signal to the counter block 102 so that the counter block 102 adjusts the value of the counter 103 associated with the main instruction to be adjusted to indicate that the hazard associated with the main instruction has been resolved (partially or completely).

[0102] Each adjustment instruction or signal may include, but is not limited to, an indication of whether the instruction is an increase instruction or a decrease instruction, a counter number, a task ID, and optionally a value to increase or decrease. In some cases, some counters 103 may only increase or decrease by a fixed amount, so the value to increase or decrease does not need to be explicitly provided. The counter update logic 404 uses the counter number and the task ID to identify a specific counter 103 in the matrix 402. The counter update logic 404 then increases or decreases the value of the identified counter 103 based on the adjustment instruction (e.g., increases or decreases the value of the counter by a specified or predetermined amount).

[0103] Although counter update logic 404 is shown as a single block that performs increments and decrements of counters 103 in matrix 402, in other examples, there may be separate increment and decrement logic blocks that receive and process increment and decrement instructions, respectively. In these cases, the adjustment instruction may not specify whether the adjustment is an increment or decrement, as this may be automatically determined based on which logic block receives the adjustment instruction.

[0104] The state logic 406 is configured to generate counter state information indicating the state of the counters 103 in the matrix 402, and provide the counter state information to at least the queue 110. This allows the queue 110 to determine whether the secondary instruction is stalled because one or more hazards associated with the main instruction on which it depends have not been resolved, or whether all hazards associated with the main instruction on which it depends have been resolved and the secondary instruction can be safely forwarded or issued to the instruction pipeline for execution (assuming there are no other hazards). Specifically, a secondary instruction can only be forwarded to or issued to the instruction pipeline 106 if the counter associated with the main instruction (one or more) on which it depends indicates that the hazard associated with the main instruction on which it depends has been resolved.

[0105] In some cases, the counter status information may include, for example, a bit or flag for each counter that indicates whether there is a hazard associated with the associated host instruction or whether the hazard associated with the associated host instruction has been resolved based on the value of the counter. For example, a flag value of "1" may indicate that there is a hazard associated with the host instruction, and a flag value of "0" may indicate that the hazard associated with the associated host instruction has been resolved. The flag value may be generated based on the value of the counter 103. For example, as described above, the counters 103 may be configured so that when they have one of one or more predetermined values ​​(e.g., a non-zero value), they indicate that there is a hazard associated with the associated host instruction, and when they have one of one or more other predetermined values ​​(e.g., a zero value), they indicate that the hazard associated with the associated host instruction has been resolved. In other cases, the counter status information may include the actual values ​​of the counters 103 in the matrix 402, and a receiver of the counter status information (e.g., a queue 110) is configured to determine whether there is a hazard associated with the associated host instruction based on the counter value.

[0106] In some cases, state logic 406 may be configured to periodically push counter state information to queue 110. In other cases, queue 110 may be configured to request counter state information, and state logic 406 may be configured to provide counter state information in response to such a request.

[0107] Reference now Figure 5 , which shows a method for avoiding parallel processing units (such as Figure 1 An example method 500 for detecting an inter-pipeline data hazard in a parallel processing unit (PPU) of a processor (CPU) of a processor. The method 500 begins at block 502, where the instruction decoder 104 receives a dispatch instruction to be decoded, and at block 504, the instruction decoder 104 decodes the received instruction. If the instruction decoder 104 determines (at block 506) that the decoded instruction is a host instruction (e.g., based on a host instruction field), the method 500 proceeds to block 508, where the instruction decoder 104 sends or transmits an adjustment instruction to the counter block 102, which causes the value of the counter 103 associated with the detected host instruction to be adjusted to have a value (e.g., a non-zero value) indicating the presence of a hazard associated with the host instruction. For example, the instruction decoder 104 may be configured to send or transmit an adjustment instruction to the counter block 102, which causes the value of the counter 103 associated with the detected host instruction to be increased by a predetermined amount (e.g., 8). The method 500 then proceeds to block 510.

[0108] If the instruction decoder 104 determines at block 506 that the decoded instruction is not a primary instruction (e.g., based on the primary instruction field), the method 500 proceeds to block 510, where the instruction is forwarded to the queue 110 associated with the appropriate instruction pipeline 106. Then at block 512, the queue 110 determines whether the received instruction is a secondary instruction. If (at block 512) the queue 110 determines that the received instruction is a secondary instruction, the method 500 proceeds to block 514, where the instruction is stalled if at least one of the (one or more) counters associated with the (one or more) primary instructions on which the secondary instruction depends indicates that there is a hazard associated with the primary instruction (e.g., determined based on the secondary instruction field). Once the (one or more) counters associated with the (one or more) primary instructions on which the secondary instruction depends indicate that the hazard associated with the primary instruction has been resolved, the method 500 proceeds to block 516.

[0109] If (at block 512 ) the queue 110 determines that the received instruction is not a secondary instruction, the method 500 proceeds directly to block 516 , where the instruction is forwarded to the corresponding instruction pipeline for execution. The method 500 then proceeds to block 518 .

[0110] At block 518, the monitor logic determines whether the instruction in the instruction pipeline is a primary instruction. If (at block 518) the monitor logic detects that the instruction in the instruction pipeline is a primary instruction, the method 500 proceeds to block 520, where the monitor logic determines whether the instruction pipeline has resolved the hazard associated with the primary instruction. If the monitor logic 108 determines that the instruction pipeline has resolved the hazard associated with the primary instruction, then (at block 522) the monitor logic 108 sends an adjustment signal or instruction to the counter block that causes the value of the counter associated with the primary instruction to be adjusted to have a value (e.g., zero) indicating that the hazard associated with the primary instruction has been resolved.

[0111] In the case where the number of counters per task ID is small compared to the number of inter-pipeline data hazards per task, counters may be frequently reused within a task. This may result in a situation where two primary instructions associated with the same counter are received at the instruction decoder in relatively rapid order so that the instruction decoder may adjust the value of the common counter to indicate that the second primary instruction has a hazard before all secondary instructions associated with the first primary instruction have a chance to read the counter to see that the hazard associated with the first primary instruction has been resolved when the instruction decoder forwards the second primary instruction for execution. For example, if the first and second primary instructions are associated with counter 1 and there are three instructions that depend on the first primary instruction, the value of counter 1 may be adjusted to indicate that there is a hazard associated with the first primary instruction in response to the instruction decoder forwarding the first primary instruction for execution, then the value of counter 1 may be adjusted to indicate that there is a hazard associated with the first primary instruction in response to detecting that the instruction pipeline has resolved the hazard in response to indicating that the hazard associated with the first primary instruction has been resolved, and then the value of counter 1 may be adjusted to indicate that there is a hazard associated with the second primary instruction in response to the instruction decoder forwarding the second primary instruction for execution, all before the queue of three secondary instructions have read or obtained the state of counter 1. This will result in the secondary instructions having to wait for the second primary instruction to complete. The presence of instructions that wait for the execution of instructions that occur later in the code is obviously problematic because it can cause deadlock.

[0112] Therefore, in some embodiments, to ensure that the counter 103 is not adjusted to reflect the state of the hazard associated with the subsequent major instruction until there are no secondary instructions waiting for the result of the hazard associated with the previous major instruction, the counter block 102 also includes a wait counter for each counter 103 that keeps track of the number of secondary instructions waiting for the result of the corresponding counter 103. The value of the wait counter of the counter 103 can be adjusted (e.g., increased) by the instruction decoder each time a secondary instruction that is dependent on the major instruction associated with the counter is sent to the queue for execution to indicate that there is an additional secondary instruction waiting for the result of the counter 103. When a secondary instruction that is dependent on the major instruction associated with the counter 103 is sent from the queue to the corresponding instruction pipeline 10 for execution, or when a secondary instruction that is dependent on the major instruction associated with the counter 103 is discarded by the queue (e.g., in response to determining that the instruction is invalid), the value of the wait counter of the counter can be adjusted (e.g., decreased) by the queue to indicate that one less secondary instruction is waiting for the result of the counter 103.

[0113] When a main instruction is received at the instruction decoder, the instruction decoder determines whether the wait counter for the corresponding counter 103 indicates that there is no secondary instruction waiting for the result of the counter. If the wait counter indicates that there is no secondary instruction waiting for the result of this counter, the instruction is forwarded to the appropriate queue for execution. However, if the wait counter indicates that there is at least one secondary instruction waiting for the result of the counter, the instruction decoder causes the instruction and its related instructions (e.g., instructions in the same task with the same task ID) to be descheduled (e.g., the instructions are sent back to the scheduler), otherwise, it waits until the wait counter indicates that there is no secondary instruction waiting for the result of the related counter.

[0114] Reference now Figure 6 , shows a second example PPU 600, which may be a GPU or other PPU including multiple wait counters. It should be understood that Figure 6 Only some elements of the PPU are shown, and there are Figure 6 Many other elements (such as cache, interface, etc.) are shown in FIG. Figure 1 The parallel processing unit 100, Figure 6 The PPU 600 includes a counter block 602 , an instruction decoder 604 , a plurality of instruction pipelines 106 , a monitor logic 108 , and a queue 610 for each instruction pipeline 106 .

[0115] Figure 6 The counter block 602 is connected with Figure 1 The counter block 102 is the same as above, except that Figure 6 A wait counter 630 is also included for each counter 103 to keep track of the number of instructions waiting for the result of the counter 103. In some examples, the wait counter 630 is configured so that a zero value indicates that there are no instructions waiting for the result of the counter 103, and a non-zero value indicates the number of instructions waiting for the result of the counter 103 (e.g., a wait counter value of 3 indicates that there are three instructions waiting for the result of the counter 103). However, this is merely an example, and other values ​​may be used to indicate the number of instructions waiting for the counter.

[0116] The counter block 602 is configured to adjust the value of the wait counter 630 in response to receiving a wait counter adjustment instruction or signal from the instruction decoder 104 and the queue 110; and to generate and provide wait counter status information to the instruction decoder 604 and the scheduler 616. Specifically, as described in more detail below, the instruction decoder 604 is configured to send a wait counter adjustment instruction or signal to the counter block 602, which causes the counter block 602 to adjust the value of the wait counter 630 corresponding to the specific counter 103 in response to forwarding a secondary instruction that depends on a primary instruction associated with the specific counter 103 to the queue 610, indicating that the additional secondary instruction is waiting for the result of the specific counter 103. The queue 610 is configured to send a wait counter adjustment instruction or signal to the counter block 602, which causes the counter block 602 to adjust the value of the wait counter 630 corresponding to the specific counter 103 to indicate that one less secondary instruction is waiting for the result of the specific counter 103 in response to the following situations: (i) forwarding the secondary instruction that depends on the main instruction associated with the specific counter 103 to the instruction pipeline for execution; or (ii) discarding the secondary instruction that depends on the main instruction associated with the specific counter 103.

[0117] The wait counter status information includes an indication of whether there are any sub-instructions waiting for the result of the counter, and is generated by the counter module 602 based on the value of the wait counter 630. In some cases, the wait counter status information may include a flag or bit for each wait counter, indicating whether there are any sub-instructions waiting for the result of the associated counter 103. For example, the wait counter status information may include a flag for each wait counter, wherein the flag is set to "1" to indicate that there is at least one sub-instruction waiting for the result of the associated counter, and the flag is set to "0" to indicate that there are no sub-instructions waiting for the result of the associated counter. In other cases, the wait counter status information may include the value of each wait counter 630, and the recipient of the wait counter status information is configured to determine from these values ​​whether there are any sub-instructions waiting for the result of the corresponding counter 103.

[0118] The following references Figure 7 describe Figure 6 An example implementation of the counter block 602 is shown.

[0119] Figure 6 The instruction decoder 604 and Figure 1 The instruction decoder 604 is the same as the instruction decoder 104, except that the instruction decoder 604 is also configured to send a wait counter adjustment instruction to the counter block 602 for each main instruction on which the secondary instruction depends in response to forwarding the secondary instruction to the queue 610 for execution, which causes the counter block 602 to adjust the value of a specific wait counter to indicate that the additional secondary instruction is waiting for the result of the related counter 103. Figure 6 The instruction decoder 604 is also configured to determine whether there are any secondary instructions waiting for the result of the counter associated with the main instruction before sending the decoded main instruction to the queue 610 for execution (based on the wait queue status information). If the instruction decoder 604 determines that there are no secondary instructions waiting for the counter result, the instruction decoder 604 sends the main instruction to the queue 610 for execution. However, if the instruction decoder 604 determines that there is at least one secondary instruction waiting for the counter result, the instruction decoder 604 sends a deactivation instruction or signal (which may also be referred to as a descheduling instruction or signal) to the scheduler 616 to cause the instruction (and / or its associated task) to be descheduled (or deactivated) until the wait counter status information indicates that there are no secondary instructions waiting for the result of the relevant counter. In some cases, the counter block 602 periodically pushes the wait counter status information to the instruction decoder 604 automatically. In other cases, the instruction decoder 604 is configured to request the wait counter status information from the counter block, and in response to receiving the request, the counter block 602 provides the wait counter status information to the instruction decoder 604.

[0120] Multiple instruction pipelines 106 and Figure 1 The instruction pipeline 106 is the same as that of FIG. 1 and is configured to execute received instructions.

[0121] Monitor logic 108 and Figure 1 The monitor logic 108 is the same.

[0122] Figure 6 The queue 610 in Figure 1 The same as queue 110 in, except that Figure 6 Each queue 610 is also configured to determine whether the instructions forwarded by the queue 610 to the corresponding instruction pipeline, or the instructions discarded by the queue 610 (for example, because they are invalid), are secondary instructions, and in response to the queue 610 forwarding the secondary instruction to the corresponding instruction pipeline or in response to the queue 610 discarding the secondary instruction, send a wait counter adjustment instruction or signal to the counter block 602 for each primary instruction that the secondary instruction depends on, so that the counter block 602 adjusts the value of the wait counter related to the counter associated with the primary instruction to indicate that one less secondary instruction is waiting for the result of the counter. In some cases, the wait counter adjustment instruction or signal sent by the queue 610 to the counter block 602 causes the counter block to reduce the value of the wait counter by 1.

[0123] Figure 6 The parallel processing unit 600 also includes a scheduler 616. The scheduler 616 and Figure 1The instruction scheduler 616 is similar to the scheduler 116, except that the instruction scheduler 616 is further configured to de-schedule or deactivate the identified instruction (and associated task) in response to receiving a deactivation instruction or signal for the instruction from the instruction decoder 604 until (one or more) related wait counters indicate that there are no more instructions waiting for the result of the related counter.

[0124] As described above, when a task has been sent from the scheduler 616 to the instruction fetch module 120 for execution, the task becomes an active task. When the scheduler 616 receives a descheduling or deactivation message for an instruction associated with a task, the task becomes an invalid task and cannot become an active task until the (one or more) associated wait counters indicate that there are no more instructions waiting for the corresponding counter results. The deactivation or descheduling instruction may include a task ID for identifying the task to be descheduled / deactivated, a program counter value for indicating the instruction (so that when the task is reactivated or rescheduled, the parallel processing unit can return to the instruction in the program), and information identifying the (one or more) associated wait counters to be monitored (e.g., wait counter numbers).

[0125] Reference now Figure 7 , Figure 7 Shows Figure 6 An example implementation of the counter block 602 is shown. Figure 6 The counter block 602 includes the above reference Figure 4 Describe the operation Figure 4 The matrix 402 of the counter 103, the counter update logic 404 and the state logic 406. Figure 6 The counter block 602 additionally includes a matrix 702 of wait counters 630 , wait counter update logic 704 , and wait counter state logic 706 .

[0126] The matrix 702 of wait counters 630 includes one wait counter 630 for each counter 103 in the matrix 402 of counters 103, which is used to track the number of instructions waiting for the result of the corresponding counter 103. The matrix 702 of wait counters 630 is arranged in the same manner as the matrix 402, so that the corresponding wait counter 630 can be identified using the same information (e.g., task ID and counter number) used to identify the counter 103. Specifically, the matrix 702 of wait counters includes a row for each possible task ID. For example, if Figure 7 As shown, in the case where there are three possible task IDs, the matrix 702 will have three rows. Each row includes K counters, where K is the number of counters in each row of the matrix 402 of counters 103.

[0127] The wait counter update logic 704 receives wait counter adjustment instructions or signals from the instruction decoder 604 and the queue 610, which causes the value of the wait counter 630 to be adjusted to reflect the number of instructions currently waiting for the result of the corresponding counter 103. Specifically, as described above with respect to Figure 6 As described, in response to sending a secondary instruction to queue 610 for execution, instruction decoder 604 sends a wait counter adjustment instruction or signal to counter block 602 for each primary instruction on which the secondary instruction depends so that the value of the wait counter corresponding to the counter associated with the primary instruction is adjusted to have a value indicating that an additional secondary instruction is waiting for a result of counter 103. In some examples, instruction decoder 604 is configured to send an adjustment instruction that causes counter block 602 to increase the value of wait counter 630 corresponding to counter 103 by 1.

[0128] When the queue 610 subsequently detects that the secondary instruction has been sent to the instruction pipeline 106 for execution, the queue 610 sends a wait counter adjustment instruction or signal to the counter block 602 for each primary instruction on which the secondary instruction depends, so that the value of the wait counter corresponding to the counter associated with the primary instruction is adjusted to indicate that one less secondary instruction is waiting for the result of the counter. In some examples, each queue 610 is configured to send a wait counter adjustment instruction or signal to the counter block 602, which causes the counter block 602 to reduce the value of the wait counter 630 corresponding to the counter 103 by 1.

[0129] Additionally, if the wait queue 610 subsequently detects that the secondary instruction has been discarded (e.g., because it is invalid), the queue 610 sends a wait counter adjustment instruction or signal to the counter block 602 for each primary instruction that the secondary instruction depends on, so that the value of the wait counter corresponding to the counter associated with the primary instruction is adjusted to indicate that one less secondary instruction is waiting for the result of that counter.

[0130] Each wait counter adjustment instruction or signal may include, but is not limited to, an indication of whether the instruction is an increase instruction or a decrease instruction, a counter number, a task ID, and optionally a value to increase or decrease. In some cases, one or more wait counters 630 may only increase or decrease by a fixed amount, so the value to increase or decrease does not need to be explicitly provided. The wait counter update logic 704 uses the counter number and the task ID to identify the wait counter 630 in the matrix 702. The wait counter update logic 704 then increases or decreases the value of the identified wait counter 103 based on the wait counter adjustment instruction (e.g., increases or decreases the value of the counter by a specified or predetermined amount).

[0131] Although the wait counter update logic 704 is shown as a single block that performs increments and decrements of the wait counter 103 in the matrix 702, in other examples, there may be separate increment and decrement logic blocks that receive and process wait counter increment and decrement instructions, respectively. In these cases, the wait counter adjustment instruction may not specify whether the adjustment is an increment or a decrement, as this may be automatically determined based on which logic block receives the adjustment instruction.

[0132] The wait counter state logic 706 is configured to generate wait counter state information indicating the state of the wait counters 103 in the matrix 702, and provide the wait counter state information to at least the scheduler 616 and the instruction decoder 604. Providing the wait counter state information to the instruction decoder 604 allows the instruction decoder 604 to determine whether it is safe to forward the main instruction to the queue for execution. Providing the wait counter state information to the scheduler 616 allows the scheduler 616 to determine when an instruction / task can be rescheduled.

[0133] As described above, in some cases, the wait counter status information may include a bit or flag for each wait counter, which indicates whether there are any sub-instructions waiting for the result of the corresponding counter 103. For example, a flag value of "1" may indicate that there is at least one sub-instruction waiting for the result of the corresponding counter 103, and a flag value of "0" may indicate that there are no sub-instructions waiting for the result of the corresponding counter 103. In other cases, the counter status information may include the actual value of the wait counter 630 in the matrix 702, and the receiver of the wait counter status information (e.g., the scheduler 616 and the instruction decoder 604) is configured to determine whether there are any sub-instructions waiting for the result of the counter 103 based on the counter value.

[0134] In some cases, wait counter state logic 706 may be configured to periodically push wait counter state information to scheduler 616 and instruction decoder 604. In other cases, scheduler 616 and instruction decoder 604 may be configured to request wait counter state information, and wait counter state logic 706 may be configured to provide wait counter state information in response to the request.

[0135] Reference now Figure 8 , Figure 8 A method for avoiding parallel processing units (such as Figure 6A second example method 800 for preventing inter-pipeline data hazards in a parallel processing unit (PPU) of a processor 610 is disclosed. The method 800 begins at block 802, where the instruction decoder 604 receives an instruction and, at block 804, decodes the instruction. At block 806, the instruction decoder 604 determines (e.g., based on a primary instruction field) whether the decoded instruction is a primary instruction. If (at block 806) it is determined that the instruction is a primary instruction, the method 800 proceeds to block 808, where a determination is made based on a wait counter corresponding to a counter associated with the primary instruction whether there are any secondary instructions waiting for a result of the counter. If (at block 808) it is determined that there is at least one secondary instruction waiting for a result of the associated counter, the method 800 proceeds to block 810, where the instruction decoder sends a de-schedule or deactivate instruction or signal to the scheduler 616 to cause the scheduler to deactivate the instruction / task until, at block 812, the wait counter corresponding to the counter associated with the primary instruction indicates that there are no secondary instructions waiting for a result of the associated counter. Once there are no secondary instructions waiting for the result of the associated counter, at block 814 , the instruction / task is rescheduled and the method 800 returns to block 802 .

[0136] If (at block 808) it is determined that there are no secondary instructions waiting for the result of the associated counter, it is safe to adjust the value of the counter for the new primary instruction, so the method 800 proceeds to block 816, where the instruction decoder 604 sends or transmits an adjustment instruction to the counter block, causing the value of the counter associated with the primary instruction to be adjusted to indicate that there is a hazard associated with the primary instruction. The method 800 then proceeds to block 818.

[0137] If (at block 806) it is determined that the decoded instruction is not a primary instruction, the method proceeds to block 818, where the instruction decoder determines (e.g., based on the secondary instruction field) whether the instruction is a secondary instruction. If it is determined that the instruction is a secondary instruction, at block 820, the instruction decoder 604 sends or transmits a wait counter adjustment instruction or signal to the counter block for each primary instruction on which the secondary instruction depends, which causes the value of the wait counter corresponding to the counter associated with the primary instruction to be adjusted to indicate that there is one more secondary instruction waiting for the result of the counter. The method 800 then proceeds to block 822.

[0138] If (at block 818 ) the instruction decoder 604 determines (eg, based on the minor instruction field) that the decoded instruction is not a minor instruction, then the method 800 proceeds to block 822 where the instruction is forwarded to a queue associated with the appropriate instruction pipeline. The method 800 then proceeds to block 824 .

[0139] At block 824, the queue determines whether the received instruction is a secondary instruction. If (at block 824) the queue determines that the received instruction is a secondary instruction, the method 800 proceeds to block 826, where if at least one counter associated with the (one or more) primary instructions that all of the secondary instructions depend on indicates (e.g., determined based on the secondary instruction field) that there is a hazard associated with the primary instruction (e.g., the counter is not zero), the instruction is stopped. Once the (one or more) counters associated with the (one or more) primary instructions that the secondary instruction depends on indicate that the hazard associated with the primary instruction has been resolved, the method 800 proceeds to block 828, where the queue 610 sends or transmits a wait counter adjustment signal to the counter block for the primary instruction that the secondary instruction depends on, which causes the value of the wait counter corresponding to the counter associated with the primary instruction to be adjusted to indicate that there is one less secondary instruction waiting for the result of the counter. The method 800 then proceeds to block 830.

[0140] If (at block 824 ) the queue determines that the received instruction is not a secondary instruction, the method 800 proceeds directly to block 830 , where the instruction is forwarded to the corresponding instruction pipeline 106 for execution. The method 800 then proceeds to block 832 .

[0141] At block 832, the monitor logic determines whether the instruction in the instruction pipeline is a primary instruction. If (at block 832) the monitor logic detects that the instruction in the instruction pipeline is a primary instruction, the method 800 proceeds to block 834, where the monitor logic determines whether the instruction pipeline has resolved the hazard associated with the primary instruction. If the monitor logic 108 determines that the instruction pipeline has resolved the hazard associated with the primary instruction, then (at block 836) the monitor logic 108 sends an adjustment signal or instruction to the counter block, which causes the value of the counter associated with the primary instruction to be adjusted to have a value (e.g., zero) indicating that the hazard associated with the primary instruction has been resolved.

[0142] Stalling a secondary instruction before it is about to be executed by the instruction pipeline shows an increase in performance when the primary instruction(s) upon which the secondary instruction depends will be completed quickly (e.g., when the primary instruction(s) are executed by an instruction pipeline with high throughput). Such inter-pipeline data hazards are referred to herein as low-latency inter-pipeline hazards. However, stalling a secondary instruction before it is about to be executed by the instruction pipeline shows a decrease in performance when the primary instruction(s) upon which the secondary instruction depends will be completed slowly (e.g., when the primary instruction(s) are executed by an instruction pipeline with low throughput). Such inter-pipeline data hazards may be referred to herein as high-latency inter-pipeline hazards.

[0143] Thus, in some embodiments described herein, the compiler may be configured to separately identify and mark low latency inter-pipeline data hazards and high latency inter-pipeline data hazards. Figure 1 The instructions of the present invention are described to handle low-latency data hazards (i.e., a secondary instruction that is waiting for one or more low-latency hazards to be cleared will wait in a queue ahead of the associated instruction pipeline for the low-latency hazard(s) to be cleared). However, high-latency data hazards are handled differently. Specifically, if the instruction decoder detects a secondary instruction that is waiting for one or more high-latency hazards to be cleared, instead of sending the secondary instruction to the queue for execution, the instruction is descheduled until all associated high-latency inter-pipeline hazards have been cleared.

[0144] Reference now Fig. 9 A third example PPU 900 is shown, which may be a GPU or other PPU that handles high-latency and low-latency inter-pipeline data hazards in different ways. It should be understood that Fig. 9 Only some elements of the PPU are shown, and there may be other elements within PPU 900 that are not shown. Fig. 9 Many other elements (e.g., caches, interfaces, etc.) are shown in FIG. Figure 1 The parallel processing unit 100, Fig. 9 The parallel processing unit 900 includes: a counter block 102, an instruction decoder 904, a plurality of instruction pipelines 106, a detector logic 108, and a monitor queue 910 for each instruction pipeline 106.

[0145] Fig. 9 The parallel processing unit 900 is configured to process instructions generated by a compiler, the compiler being configured to (i) identify inter-pipeline data hazards; (ii) determine whether the identified inter-pipeline data hazard is a low-latency data hazard or a high-latency data hazard (e.g., based on a throughput associated with an instruction pipeline in which a host instruction with the inter-pipeline data hazard is to be executed); and (iii) assign counters 103 from a first subset of counters 103 (e.g., low-latency counters) to low-latency inter-pipeline data hazards and assign counters 103 from a second, different subset of counters 103 (e.g., high-latency counters) to high-latency inter-pipeline data hazards. Accordingly, the instructions generated by Fig. 9 The instructions processed by the parallel processing unit 900 include information (e.g., inserted by a compiler at build time) identifying a primary instruction (i.e., an instruction on which at least one other instruction in another instruction pipeline depends), a secondary instruction (i.e., an instruction that depends on at least one primary instruction in another pipeline), their associated counters, and information indicating whether the associated hazard is a low-latency hazard or a high-latency hazard.

[0146] For example, Fig.10As shown, if there are three counters numbered 1 to 3 (e.g., for each possible task ID), a first set of counters formed by counters 1 and 2 may be assigned to low latency inter-pipeline data hazards, and a second set of counters formed by counter 3 may be assigned to high latency inter-pipeline data hazards. PPU 900 may use the counter numbers associated with a primary or secondary instruction to determine whether the primary or secondary instruction is associated with a low latency inter-pipeline data hazard or a high latency inter-pipeline data hazard, and process it accordingly.

[0147] Back to Fig. 9 , counter block 102 and Figure 1 The counter block 102 is the same as the counter block 102 of the embodiment of the present invention, except that the counter block 102 is also configured to provide counter status information to the scheduler 916 and the instruction decoder 904. Specifically, the counter block 102 includes a plurality of counters 103 for tracking inter-pipeline data hazards. The counter block 102 receives an adjustment instruction or signal from the instruction decoder 904 to adjust the value of the counter to indicate that there is a hazard associated with the associated main instruction when the instruction decoder forwards the main instruction to the queue 910 for execution; and the counter block 102 subsequently receives an adjustment instruction or signal from the monitor logic 108 to adjust the value of the counter to indicate that the hazard associated with the associated main instruction has been resolved when the monitor logic 108 detects that the instruction pipeline has resolved the hazard associated with the main instruction. The counter block also provides counter status information to the queue 910, the scheduler 916, and the instruction decoder 904, and the counter status information indicates, for each counter, whether there is a hazard associated with the main instruction associated therewith. The fact that one or more counters 103 are designated for low latency inter-pipeline data hazards and one or more counters 103 are designated for high latency inter-pipeline data hazards does not affect the operation of counter block 102 .

[0148] Instruction decoder 904 generally corresponds to Figure 1 The instruction decoder 104 of the processor 100 is configured to receive instructions, decode the received instructions, forward the decoded instructions to the queue 910 associated with the appropriate pipeline for execution; and in response to forwarding a main instruction (whether related to a low-latency inter-pipeline data hazard or a high-latency inter-pipeline data hazard) to the queue 910 for execution, send an adjustment instruction to the counter block so that the value of the counter associated with the main instruction is adjusted to indicate that there is a hazard associated with the main instruction (for example, sending an adjustment instruction to increase the value of the counter associated with the main instruction by a predetermined amount (for example, 8)).

[0149] However, with Figure 1 The instruction decoder 104 automatically forwards all decoded sub-instructions to the queue 910 for execution. Fig. 9The instruction decoder 904 is configured to determine (e.g., based on a counter number associated therewith) whether the secondary instruction is associated with at least one high latency data hazard before forwarding the decoded secondary instruction to the queue 910 for execution. If it is determined that the secondary instruction is not associated with any high latency data hazard, the instruction decoder 904 forwards the secondary instruction to the queue 910 for execution.

[0150] However, if it is determined that the secondary instruction is associated with at least one high-latency data hazard, the instruction decoder 904 determines whether each high-latency inter-pipeline data hazard associated with the secondary instruction has been resolved based on the counter status information provided by the counter block 102. If the counter status information indicates that all high-latency data hazards associated with the secondary instruction have been resolved (e.g., any high-latency counter associated with the secondary instruction indicates that the high-latency data hazard has been resolved), the secondary instruction is forwarded to the queue 910 for execution. However, if the counter status information indicates that at least one of the high-latency hazards associated with the secondary instruction has not been resolved, a deactivation instruction or signal is sent to the scheduler 916 to cause the scheduler to deactivate the instruction (and / or the task associated therewith). The deactivation instruction may include information identifying the instruction (e.g., the value of the program counter) and information identifying the high-latency counter to be monitored, and in the case where the instruction is part of a task, the deactivation message may also include information identifying the task (e.g., a task ID).

[0151] Multiple instruction pipelines 106 and Figure 1 The instruction pipeline 106 is the same as that of FIG. 1 and is thus configured to execute received instructions.

[0152] Monitor logic 108 and Figure 1 The monitor logic 108 is the same.

[0153] Queue 910 and Figure 1 The same as queue 110, except that Fig. 9 The queue 910 is configured to stall a secondary instruction only when the low latency counter associated with the secondary instruction (and not other counters) indicates that there is a hazard associated with the associated major instruction.

[0154] Fig. 9 The parallel processing unit 900 also includes a scheduler 916. The scheduler 916 and Figure 1 The scheduler 116 is the same as Fig. 9 The scheduler 916 is also configured to, in response to receiving a deactivation instruction or signal for a secondary instruction from the instruction decoder 904, deschedule or deactivate the identified secondary instruction (and associated task) until the high latency counter associated with the secondary instruction indicates that (one or more) high latency data hazards have been resolved.

[0155] As described above, when a task has been sent from the scheduler 916 to the instruction fetch module for processing, the task becomes a valid task. When the scheduler 916 receives a descheduling or deactivation message about an instruction, the associated task becomes an invalid task and cannot become a valid task until (one or more) related high latency counters indicate that the related high latency hazard has been resolved. The deactivation or descheduling instruction may include information identifying the instruction to be descheduled / deactivated (e.g., a program counter value), information identifying the task associated with the instruction (e.g., a task ID), and information identifying the high latency counter that needs to be monitored (e.g., a counter number).

[0156] Reference now Fig.11 , showing a method for avoiding parallel processing units (such as, Figure 1 A third example method 1100 for detecting inter-pipeline data hazards in a parallel processing unit 900 of FIG. 1 is provided. The method 1100 begins at block 1102, where the instruction decoder 904 receives an instruction and decodes the instruction (at block 1104). At block 1106, the instruction decoder 904 determines (e.g., based on a secondary instruction field) whether the decoded instruction is a secondary instruction. If (at block 1106) it is determined that the instruction is a secondary instruction, the method 1100 proceeds to block 1108, where it is determined whether the instruction is associated with at least one high-latency inter-pipeline hazard based on a number of counters associated with each primary instruction that is dependent on. If (at block 1108) it is determined that the secondary instruction is associated with at least one high-latency inter-pipeline data hazard, the method 1100 proceeds to block 1110, where the instruction decoder 904 determines whether the associated high-latency counters on which the secondary instruction depends indicate the presence of a high-latency hazard associated with the primary instruction. If it is determined (at block 1110) that the relevant high latency counter indicates that there is a high latency hazard associated with the host instruction, the method 1100 proceeds to block 1112, where the instruction decoder 904 sends a de-schedule or deactivate instruction or message to the scheduler, causing the scheduler to deactivate the instruction / task until the relevant high latency counter(s) indicate that the high latency hazard associated with the host instruction(s) has been resolved at block 1114. Once the relevant high latency counter(s) indicate that the high latency hazard associated with the host instruction(s) has been resolved, the instruction / task is rescheduled at block 1116 and the method 1100 returns to block 1102.

[0157] If it is determined (at block 1108 ) that the secondary instruction is not associated with any high latency counters, or it is determined (at block 1110 ) that the associated high latency hazard has been resolved, the method proceeds to block 1118 .

[0158] If it is determined (at block 1106) that the decoded instruction is not a secondary instruction, then (at block 1118) the instruction decoder determines (e.g., based on the primary instruction field) whether the instruction is a primary instruction. If it is determined that the instruction is a primary instruction, then at block 1120, the instruction decoder 904 sends or transmits a counter adjustment instruction or signal to the counter block so that the value of the counter associated with the primary instruction is adjusted to indicate that there is a hazard associated with the primary instruction. The method (1100) then proceeds to block 1122.

[0159] If (at block 1118) the instruction decoder determines (e.g., based on the primary instruction field) that the decoded instruction is not a primary instruction, the method 1100 proceeds to block 1122, where the instruction is forwarded to a queue associated with the appropriate instruction pipeline. Then at block 1124, the queue 910 determines whether the received instruction is a secondary instruction. If (at block 1124) the queue 910 determines that the received instruction is a secondary instruction, the method 1100 proceeds to block 1126, and the instruction is stalled until the (one or more) low latency counters associated with the (one or more) primary instructions on which the secondary instruction depends indicate that the low latency hazard associated with the primary instruction has been resolved (e.g., the counter is zero). Once the (one or more) low latency counters associated with the (one or more) primary instructions on which the secondary instruction depends indicate that the low latency hazard associated with the primary instruction has been resolved, the method 1100 proceeds to block 1128.

[0160] If (at block 1124 ) the queue 910 determines that the received instruction is not a secondary instruction, the method 1100 proceeds directly to block 1128 , where the instruction is forwarded to the corresponding instruction pipeline for execution.

[0161] At block 1130, the monitor logic determines whether the instruction in the instruction pipeline is a primary instruction. If (at block 1130) the monitor logic determines that the instruction in the instruction pipeline is a primary instruction, the method 1100 proceeds to block 1132, where the monitor logic determines whether the instruction pipeline has resolved the hazard associated with the primary instruction. If the monitor logic 108 determines that the instruction pipeline has resolved the hazard associated with the primary instruction, then (at block 1134) the monitor logic 108 sends an adjustment signal or instruction to the counter block so that the value of the counter associated with the primary instruction is adjusted to indicate that the hazard associated with the primary instruction has been resolved.

[0162] Although not shown, in other examples, reference Figures 6 to 8 The wait counters described can be compared with the Figures 9 to 11 The described low latency / high latency processing of inter-pipeline data hazards is combined. In these examples, there may be only one wait counter for each low latency counter.

[0163] Reference now Fig.12, showing the method for generating Figure 1 and Figure 2 An example method 1200 for executing instructions by a parallel processing unit 100, 600 of a program is provided. The method 1200 begins at block 1202, where a set of related instructions (e.g., instructions that form a program) is received (e.g., at a compiler). The set of related instructions can be in any suitable format. For example, in some cases, the set of related instructions can be in machine language. Once the set of related instructions is received, the method 1200 proceeds to block 1204.

[0164] At block 1204, the instructions are analyzed (e.g., by a compiler) to identify inter-pipeline data hazards in the set of related instructions. As described above, an inter-pipeline data hazard exists when an instruction to be executed in one pipeline can only be safely executed after another instruction executed in another instruction pipeline has performed a data action (e.g., read or write data). Three types of data hazards may generally occur: WAR hazards, RAW hazards, and WAW hazards.

[0165] Instructions that perform actions are referred to herein as primary instructions, and instructions that wait for actions to be performed by the primary instructions are referred to as secondary instructions. For example, in a WAR hazard, a read instruction is a primary instruction and a write instruction is a secondary instruction; in a RAW hazard, a write instruction is a primary instruction and a read instruction is a secondary instruction; and in a WAW hazard, a first write instruction is a primary instruction and a second write instruction is a secondary instruction. Secondary instructions are considered to be dependent on a primary instruction. As described above, there may be more than one secondary instruction that is dependent on the same primary instruction.

[0166] In some cases, the compiler may be configured to identify inter-pipeline data hazards in related instructions (and their primary and secondary instructions) by analyzing the operands of the instructions to identify WAR, RAW, and WAW hazards. In other cases, the group of instructions may include information indicating intra-pipeline data hazards in the group of instructions (e.g., manually inserted by a programmer or developer), and the compiler may be configured to identify inter-pipeline data hazards (and their primary and secondary instructions) based on the information. Once the inter-pipeline data hazards have been identified, the method 1200 proceeds to block 1206.

[0167] At block 1206, each host instruction for which inter-pipeline data hazards are identified is assigned (e.g., by a compiler) a counter (e.g., a counter number) from a plurality of counters (e.g., counters 103) to track the identified inter-pipeline data hazards. As described above, in some cases, counters may be assigned to host instructions in a manner that maximizes the time between reuses of counters (e.g., in a round-robin manner). Once a counter has been assigned to each host instruction, method 1200 proceeds to block 1208.

[0168] At block 1208, computer executable instructions are generated (e.g., by a compiler) for each host instruction, such that the computer executable instructions include information identifying the host instruction as a host instruction, and information identifying a counter assigned to the host instruction. Figure 2 As described, in some cases, the generated computer executable instruction may include a main instruction field, which includes information indicating that the computer executable instruction is a main instruction, and information identifying a counter assigned to the main instruction. For example, the main instruction field can be configured to store a number, and when the number is a predetermined value (e.g., zero), it indicates that the instruction is not a main instruction, and when the number is not a predetermined value (e.g., a zero value), it indicates that the instruction is a main instruction, and the number represents the number of the counter associated with the main instruction. Once the computer readable instructions are generated for the main instruction, the method 1200 proceeds to block 1210.

[0169] At block 1210, computer executable instructions are generated (e.g., by a compiler) for each secondary instruction, such that the computer executable instructions include information identifying the computer executable instruction as a secondary instruction and information identifying a counter assigned to the corresponding primary instruction. Figure 2 As described, in some cases, the generated computer executable instruction includes a secondary instruction field, which includes information indicating that the computer executable instruction is a secondary instruction and information identifying a counter assigned to the corresponding primary instruction. For example, the secondary instruction field can be configured to hold a bit mask, wherein each bit of the bit mask corresponds to a counter of a plurality of counters, and when a bit of the mask is set, it indicates that the instruction is a secondary instruction that is dependent on a primary instruction associated with the corresponding counter. Once the computer readable instructions are generated for the secondary instruction, the method 1200 proceeds to block 1212.

[0170] At block 1212, the computer executable instructions generated in blocks 1208 and 1210 are loaded into or accessible by a memory of a parallel processing unit.

[0171] Despite Fig.12 In example method 1200, all inter-pipeline data hazards are identified and then counters are assigned thereto and executable instructions generated therefor for primary and secondary instructions. In other examples, after identifying each inter-pipeline data hazard, counters may be assigned to the primary instruction and computer executable instructions generated for the primary and secondary instructions to include information indicating that they are so and identifying the counters assigned to the primary instruction.

[0172] When Fig.12 The instructions generated by method 1200 are intended to be executed by a parallel processing unit (such as a processor) that is configured to handle low-latency inter-pipeline data hazards differently from high-latency inter-pipeline data hazards. Fig. 9When executing a parallel processing unit of a processor (i.e., a processor), counters for tracking data hazards can be divided into two groups, and counters from one group can be assigned to host instructions with low-latency data hazards, and counters from another group can be assigned to host instructions with high-latency data hazards. As described above, the parallel processing unit can then determine whether to process a hazard as a low-latency hazard or a high-latency hazard based on the counters assigned to the corresponding host instructions.

[0173] Reference now Fig.13 , which shows an example method of assigning counters to host instructions (block 1206), wherein host instructions with high latency data hazards are assigned counters from one set of counters, and host instructions with low latency data hazards are assigned counters from another set of counters. Method 1206 starts from block 1302, wherein it is determined whether the inter-pipeline data hazard is a low latency data hazard or a high latency data hazard for the identified inter-pipeline data hazard. The determination can be based on the throughput of the instruction pipeline (e.g., instruction pipeline 106) that will execute the host instruction with the identified inter-pipeline data hazard. The throughput provides an indication of the speed at which the instruction pipeline processes instructions. Generally, the higher the throughput, the faster the instruction pipeline processes instructions. An instruction pipeline having a throughput greater than or equal to a predetermined threshold can be considered a high throughput instruction pipeline, and a host instruction to be executed by the high throughput instruction pipeline can be considered to be associated with a low latency interval pipeline data hazard. Conversely, an instruction pipeline having a throughput less than a predetermined threshold can be considered a low throughput instruction pipeline, and a host instruction to be executed by the low throughput instruction pipeline can be considered to be associated with a high latency inter-pipeline data hazard. Conversely, an instruction pipeline having a throughput less than a predetermined threshold can be considered a low throughput instruction pipeline, and a host instruction to be executed by the low throughput instruction pipeline can be considered to be associated with a high latency inter-pipeline data hazard. Determining which instruction pipeline will execute a subject instruction may be based on the type of subject instruction, which may be identified, for example, by an operation code ("opcode") of the subject instruction.

[0174] If it is determined that the host instruction is associated with a low latency hazard, the method 1206 proceeds to block 1304, where the host instruction is assigned (e.g., by the compiler) a counter from the first set of counters (e.g., counters 0 to 3). The method 1206 then proceeds to block 1308. However, if it is determined that the host instruction is associated with a high latency hazard, the method 1206 proceeds to block 1306, where the host instruction is assigned (e.g., by the compiler) a counter from the second set of counters (e.g., counters 4 to 7). The method 1306 then proceeds to block 1308.

[0175] At block 1308, it is determined whether there are more identified inter-pipeline data hazards. If there is at least one identified inter-pipeline data hazard, the method returns to block 1302. If there are no more identified inter-pipeline data hazards (i.e., all host instructions have been assigned counters), the method 1206 ends.

[0176] Fig.14A computer system is shown in which the parallel processing units 100, 600, 900 described herein may be implemented. The computer system includes a CPU 1402, a GPU 1404, a memory 1406, and other devices 1414, such as a display 1416, a speaker 1418, and a camera 1420. A parallel processing unit 1410 (corresponding to any of parallel processing units 100, 600, or 900) is shown as being implemented within the GPU 1404. In other examples, the parallel processing unit 1410 may be implemented within the CPU 1402. The components of the computer system may communicate with each other via a communication bus 1422.

[0177] Figure 1 , 3 , 4, 6, 7 and 9 of the parallel processing unit, queue and counter block are shown as including many functional blocks. This is only schematic and is not intended to define the strict division between the different logical elements of these entities. Each functional block can be provided in any suitable manner. It should be understood that the intermediate values ​​formed by the elements of the parallel processing unit, queue or counter block described herein do not need to be physically generated by the parallel processing unit, queue or counter block anywhere, and can only represent the logical value of the processing performed by the parallel processing unit, queue or counter block between its input and output.

[0178] The parallel processing units, queues and / or counter blocks described herein may be implemented with hardware on an integrated circuit. The parallel processing units described herein may be configured to perform any of the methods described herein. Typically, any of the above functions, methods, techniques or components may be implemented with software, firmware, hardware (e.g., fixed logic circuits) or any combination thereof. The terms "module", "function", "component", "element", "unit", "block" and "logic" may be used here to generally represent software, firmware, hardware or any combination thereof. In the case of software implementation, a module, function, component, element, unit, block or logic represents a program code that performs a specified task when executed on a processor. The algorithms and methods described herein may be performed by one or more processors that execute a code that causes (one or more) processors to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disk, flash memory, hard disk storage, and other devices that can use magnetic, optical and other technologies to store instructions or other data and can be accessed by a machine.

[0179] The terms computer program code and computer readable instructions as used herein refer to any kind of executable code for a processor, including code expressed in a machine language, an interpreted language, or a scripting language. Executable code includes binary code, machine code, byte code, code that defines an integrated circuit (such as a hardware description language or a netlist), and code expressed in a programming language code (e.g., C, Java, or OpenCL). Executable code can be any type of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, executed, for example, in a virtual machine or other software environment, causes a processor of a computer system that supports code executables to perform tasks specified by the code.

[0180] A processor, parallel processing unit, computer or computer system may be any kind of device, machine or special circuit or a collection or part thereof having the capability to enable it to perform instruction processing. A processor may be any kind of general or special processor, such as a CPU, GPU, system on chip, state machine, media processor, application specific integrated circuit (ASIC), programmable logic array, field programmable gate array, programmable gate array (FPGA), etc. A computer or computer system may include one or more processors.

[0181] It is also intended to cover software (such as HDL (Hardware Description Language) software) that defines the configuration of hardware as described herein to perform the desired functions, such as for designing integrated circuits or for configuring programmable chips. In other words, a computer readable storage medium may be provided having encoded thereon computer readable program code in the form of an integrated circuit definition data set that, when processed in an integrated circuit manufacturing system, configures the system to manufacture a system configured to perform the methods described herein, or to manufacture a processor or parallel processing unit including any of the apparatus described herein. The integrated circuit definition data set may be, for example, an integrated circuit description.

[0182] A method of manufacturing a parallel processing unit, queue and / or counter block as described herein at an integrated circuit manufacturing system may be provided. An integrated circuit definition data set may be provided which, when processed in an integrated circuit manufacturing system, causes the method of manufacturing a parallel processing unit, queue and / or counter block to be performed.

[0183] The integrated circuit definition data set may be in the form of computer code, for example, as a netlist, code for configuring a programmable chip, as a hardware description language that defines any level of integrated circuit, including register transfer level (RTL) code, as a high-level circuit representation such as Verilog or VHDL, and as a low-level circuit representation such as OASIS (RTM) and GDSII. A higher-level representation that logically defines the integrated circuit (e.g., RTL) may be processed at a computer system configured to generate a manufacturing definition of the integrated circuit in the context of a software environment that includes definitions of circuit elements and software for combining those elements to generate the integrated circuit defined by the representation. As is typically the case with software that is executed at a computer system to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to cause the computer system configured to generate a manufacturing definition integrated circuit to execute the code that defines the integrated circuit, thereby generating a manufacturing definition of the integrated circuit.

[0184] Now refer to Fig.15 An example of processing an integrated circuit definition data set at an integrated circuit manufacturing system to configure the system to manufacture a parallel processing unit (eg, parallel processing unit 100 , 600 , or 900 ) is described.

[0185] Fig.15 An example of an integrated circuit (IC) manufacturing system 1502 configured to manufacture a parallel processing unit (e.g., parallel processing unit 100, 600, or 900) as described in any of the examples herein is shown. In particular, IC manufacturing system 1502 includes a layout processing system 1504 and an integrated circuit generation system 1506. IC manufacturing system 1502 is configured to receive an IC definition data set (e.g., defining a parallel processing unit (e.g., parallel processing unit 100, 600, or 900) as described in any of the examples herein), process the IC definition data set, and generate an IC (e.g., embodying a parallel processing unit (e.g., parallel processing unit 100, 600, or 900)) according to the IC definition data set. Processing of the IC definition data set configures IC manufacturing system 1502 to manufacture an integrated circuit embodying a parallel processing unit (e.g., parallel processing unit 100, 600, or 900) as described in any of the examples herein.

[0186] The layout processing system 1504 is configured to receive and process an IC definition data set to determine a circuit layout. Methods for determining a circuit layout based on an IC definition data set are known in the art and may involve, for example, synthesizing RTL code to determine a gate level representation of a circuit to be generated, for example, in terms of logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining the location information of the logic components, the circuit layout can be determined from the gate level representation of the circuit. This can be done automatically or with user participation to optimize the circuit layout. When the layout processing system 1504 has determined the circuit layout, it can output a circuit layout definition to the IC generation system 1506. The circuit layout definition can be, for example, a circuit layout description.

[0187] As is known in the art, IC generation system 1506 generates an IC according to the circuit layout definition. For example, IC generation system 1506 can implement a semiconductor device manufacturing process to generate an IC, which can involve a multi-step sequence of photolithography and chemical processing steps in which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition can be in the form of a mask that can be used in a photolithography process to generate an IC according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1506 can be in the form of a computer-readable code that IC generation system 1506 can use to form a suitable mask for generating an IC.

[0188] The different processes performed by IC manufacturing system 1502 can all be located in one location (e.g., implemented by one party). Alternatively, IC manufacturing system 1502 can be a distributed system such that some processes can be performed at different locations and can be performed by different parties. For example, some stages of: (i) synthesizing RTL code representing an IC definition data set to form a gate level representation of a circuit to be generated, (ii) generating a circuit layout based on the gate level representation, (iii) forming a mask based on the circuit layout, and (iv) using the mask to manufacture the integrated circuit can be performed at different locations and / or by different parties.

[0189] In other examples, an integrated circuit definition data set at an integrated circuit manufacturing system may configure the system to manufacture a parallel processing unit (e.g., parallel processing unit 100, 600, or 900) without processing the IC definition data set to determine the circuit layout. For example, the integrated circuit definition data set may define a configuration of a reconfigurable processor such as an FPGA, and processing of the data set may configure the IC manufacturing system to generate a reconfigurable processor having the defined configuration (e.g., by loading the configuration data into the FPGA).

[0190] In some embodiments, when the integrated circuit manufacturing definition data set is processed in the integrated circuit manufacturing system, the integrated circuit manufacturing definition data set can enable the integrated circuit manufacturing system to generate the device as described herein. Fig.15 An integrated circuit manufacturing system in the manner described can enable the manufacture of devices as described herein.

[0191] In some examples, the integrated circuit definition data set may include software that runs on or in conjunction with hardware defined on the data set. Fig.15 In the example shown, the IC generation system can also be configured by the integrated circuit definition data set to load firmware onto the integrated circuit when manufacturing the integrated circuit according to the program code defined in the integrated circuit definition data set, or otherwise provide program code to the integrated circuit for use with the integrated circuit.

[0192] Compared to known implementations, the implementation of the concepts set forth in this application in devices, apparatuses, modules and / or systems (and in the methods implemented herein) can cause performance improvements. Performance improvements can include one or more of increased computing performance, reduced delays, increased throughput and / or reduced power consumption. During the manufacture of such devices, devices, modules and systems (e.g., in integrated circuits), performance improvements can be weighed against physical implementations to improve manufacturing methods. For example, performance improvements can be weighed against layout areas to match the performance of known implementations but using less silicon. This can be accomplished, for example, by reusing functional blocks in a serialized manner or sharing functional blocks between elements of devices, manufacturing, modules and / or systems. On the contrary, the concepts proposed in this application that can improve the physical implementation of devices, devices, modules and systems (e.g., reduced silicon area) can be weighed for improved performance. This can be accomplished, for example, by manufacturing multiple instances of modules within a predefined area budget.

[0193] The applicant hereby independently discloses each individual feature described herein and any combination of two or more such features, so that such features or combinations can be based on the present description as a whole according to the general common sense of those skilled in the art, regardless of whether these features or combinations of features solve any problems disclosed herein. In view of the foregoing description, it will be apparent to those skilled in the art that various modifications can be made within the scope of the present invention.

[0194] Further examples are now provided by clause.

[0195] Item 1: A computer-implemented method 1200 for generating computer executable instructions for a parallel processing unit, the method 1200 comprising, by a processor: receiving 1202 a plurality of related instructions; identifying 1204 data hazards in the plurality of related instructions, each data hazard comprising a primary instruction and one or more secondary instructions; assigning 1206 a counter from a plurality of counters to each primary instruction for tracking the identified data hazards; generating 1208 a computer executable instruction for each primary instruction, the computer executable instruction comprising information indicating that the computer executable instruction is a primary instruction and information identifying the counter assigned to the primary instruction; and generating 1210 a computer executable instruction for each secondary instruction, the computer executable instruction comprising information indicating that the computer executable instruction is a secondary instruction and information identifying the counter assigned to the corresponding primary instruction; and loading 1212 the computer executable instructions into the parallel processing unit.

[0196] Clause 2: The method 1200 described in Clause 1, wherein each computer executable instruction includes a main instruction field and a secondary instruction field, the main instruction field is configured to include information indicating that the computer executable instruction is a main instruction and information identifying a counter assigned to the main instruction, and the secondary instruction field is configured to include information indicating that the computer executable instruction is a secondary instruction and information identifying a counter assigned to the corresponding main instruction.

[0197] Clause 3: The method 1200 of clause 2, wherein the primary instruction field is configured to hold a number and indicates that the instruction is not a primary instruction when the number is a predetermined value and indicates that the instruction is a primary instruction when the number is not a predetermined value, and the number represents a number assigned to a counter of the primary instruction.

[0198] Clause 4: The method 1200 of clause 2 or clause 3, wherein the secondary instruction field is configured to hold a bit mask, wherein each bit of the bit mask corresponds to a counter of the plurality of counters and when the mask is set indicates that the instruction is a secondary instruction that is dependent on the primary instruction assigned to the corresponding counter.

[0199] Clause 5: Method 1200 of any of clauses 2 to 4, wherein counters for tracking identified data hazards are assigned to main instructions, including determining 1302 whether the identified data hazard is a high-latency data hazard or a low-latency data hazard, and when the identified data hazard is a high-latency data hazard, allocating 1304 counters from a first subset of multiple counters to the main instruction, and when the identified data hazard is a low-latency data hazard, allocating 1306 counters from a second subset of multiple counters to the main instruction.

[0200] Clause 6: The method 1200 of clause 5, wherein determining whether the identified data hazard is a high latency data hazard or a low latency data hazard is based on a throughput associated with an instruction pipeline used to execute a host instruction of the identified data hazard.

[0201] Clause 7: The method 1200 of Clause 6, wherein if the throughput associated with the instruction pipeline used to execute the main instruction for the identified data hazard is greater than or equal to a predetermined threshold, then the identified data hazard is a high-latency data hazard, otherwise, the identified data hazard is a low-latency data hazard.

[0202] Clause 8: The method 1200 of clause 6 or clause 7, wherein the instruction pipeline used to execute the identified data-hazardous host instruction is based on a type of the host instruction.

[0203] Clause 9: The method 1200 of any of clauses 1 to 8, wherein the parallel processing unit comprises a plurality of parallel instruction pipelines, and identifying data hazards within the plurality of related instructions comprises identifying correlations among pipeline data hazards within the plurality of related instructions.

[0204] Clause 10: The method 1200 of any of clauses 1-9, wherein the parallel processing unit is configured to track data hazards using a counter identified in the computer-readable instructions.

[0205] Clause 11: Method 1200 of any of clauses 1 to 10, wherein the parallel processing unit is configured to: in response to a decoder of the parallel processing unit outputting a primary instruction for execution, adjust a counter assigned to the primary instruction to indicate a data hazard associated with the primary instruction; in response to detecting that the data hazard associated with the primary instruction has been resolved, adjust a counter assigned to the primary instruction to indicate that the data hazard has been resolved; and in response to the decoder outputting a secondary instruction for execution, stall the secondary instruction if at least one counter assigned to the primary instruction on which the secondary instruction depends indicates that there is a data hazard associated with the primary instruction.

[0206] Clause 12: The method 1200 of any of clauses 1 to 11, wherein the parallel processing unit comprises: a plurality of counters 103; a plurality of queues 110, 610, 910, each queue 110, 610, 910 preceding one of the plurality of instruction pipelines 106; an instruction decoder 104, 604, 904 configured to: decode a received instruction; in response to determining that the decoded instruction is a primary instruction on which at least one other instruction depends, cause a value of a counter 103 of the plurality of counters assigned to the primary instruction to be adjusted to indicate that there is a hazard associated with the primary instruction; and forward the decoded instruction to the plurality of queues 110, 610, 910. 0, 910; and a monitor logic 108, configured to monitor the multiple instruction pipelines 106, and in response to detecting that the instruction pipeline 106 has resolved the hazard associated with the main instruction, the value of the counter assigned to the main instruction is adjusted to indicate that the hazard associated with the main instruction has been resolved; wherein each queue 110, 610, 910 is configured to, in response to receiving a secondary instruction that depends on one or more main instructions, stall the execution of the secondary instruction by the associated instruction pipeline (106) if a counter 103 among the multiple counters assigned to the main instruction on which the secondary instruction depends indicates that there is a hazard associated with the main instruction.

[0207] Clause 13: The method of any of clauses 1 to 12, wherein a primary instruction is an instruction that one or more instructions depend on, and a secondary instruction is an instruction that depends on the one or more primary instructions.

[0208] Clause 14: Computer program code for performing the method of any of clauses 1 to 13.

[0209] Clause 15: A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, which when executed at a computer system, cause the computer system to perform the method of any one of clauses 1 to 13.

Claims

1. A computer-implemented method for configuring a parallel processing unit, the method comprising, by a processor: receiving multiple related instructions; identifying data hazards in the plurality of related instructions, each data hazard comprising a primary instruction and one or more secondary instructions; allocating a counter of a plurality of counters to each primary instruction to track the identified data hazard; generating computer executable instructions for each host instruction, the computer executable instructions including information indicating that the computer executable instruction is a host instruction and information identifying a counter assigned to the host instruction; generating a computer executable instruction for each secondary instruction, the computer executable instruction including information indicating that the computer executable instruction is a secondary instruction and information identifying a counter assigned to the corresponding primary instruction; and The computer executable instructions are loaded into a parallel processing unit.

2. The method according to claim 1, wherein: A primary instruction is an instruction that one or more instructions depend on, and a secondary instruction is an instruction that depends on one or more primary instructions.

3. The method of claim 1 or claim 2, further comprising determining whether each identified data hazard is a high-latency data hazard or a low-latency data hazard; and wherein, When the identified data hazard is a low latency data hazard, the allocated counter is from a first subset of the plurality of counters, and when the identified data hazard is a high latency data hazard, the allocated counter is from a second subset of the plurality of counters.

4. The method according to claim 3, wherein: The parallel processing unit is configured to track data hazards using counters identified in the computer-executable instructions and to handle low latency data hazards differently from high latency data hazards based on the counters assigned to corresponding host instructions.

5. The method according to claim 1 or claim 2, wherein: Each computer executable instruction includes a main instruction field and a secondary instruction field, wherein the main instruction field is configured to include information indicating that the computer executable instruction is a main instruction and information identifying a counter assigned to the main instruction, and the secondary instruction field is configured to include information indicating that the computer executable instruction is a secondary instruction and information identifying a counter assigned to the corresponding main instruction.

6. The method according to claim 5, wherein: The primary instruction field is configured to hold a number, and when the number is a predetermined value, the number indicates that the instruction is not a primary instruction, and when the number is not the predetermined value, the number indicates that the instruction is a primary instruction, and the number represents a number of the counter assigned to the primary instruction.

7. The method according to claim 5, wherein: The secondary instruction field is configured to hold a bit mask, wherein each bit of the bit mask corresponds to a counter of the plurality of counters, and when a bit in the mask is set, the bit indicates that the instruction is a secondary instruction that is dependent on the primary instruction to which the corresponding counter is assigned.

8. The method according to claim 5, wherein: Allocating counters to a host instruction to track an identified data hazard includes determining whether the identified data hazard is a high-latency data hazard or a low-latency data hazard, allocating counters from a first subset of the plurality of counters to the host instruction when the identified data hazard is a high-latency data hazard, and allocating counters from a second subset of the plurality of counters to the host instruction when the identified data hazard is a low-latency data hazard.

9. The method according to claim 1 or claim 2, wherein: Determining whether the identified data hazard is a high latency data hazard or a low latency data hazard is based on a throughput associated with an instruction pipeline used to execute the subject instruction of the identified data hazard.

10. The method according to claim 9, wherein: If the throughput associated with the instruction pipeline used to execute the subject instruction of the identified data hazard is greater than or equal to a predetermined threshold, the identified data hazard is a high latency data hazard, otherwise, the identified data hazard is a low latency data hazard.

11. The method according to claim 9, wherein: The instruction pipeline used to execute the host instruction for the identified data hazard is based on a type of the host instruction.

12. The method according to claim 1 or claim 2, wherein: The parallel processing unit includes a plurality of parallel instruction pipelines, and identifying data hazards within the plurality of related instructions includes identifying inter-pipeline data hazards within the plurality of related instructions.

13. The method according to claim 1 or claim 2, wherein: The parallel processing unit is configured to track data hazards using the counter identified in the computer-executable instructions.

14. The method of claim 1 or claim 2, further comprising executing the computer executable instructions at the parallel processing unit.

15. The method according to claim 14, wherein: Executing the computer executable instructions at the parallel processing unit comprises: decoding the computer-executable instructions at an instruction decoder of the parallel processing unit; responsive to determining at the instruction decoder that the decoded instruction is a subject instruction, causing a value of the counter associated with the subject instruction to be adjusted to indicate that a hazard associated with the subject instruction exists; forwarding decoded instructions from the instruction decoder to a queue of a plurality of queues, each queue being configured to receive instructions to be executed by one of a plurality of instruction pipelines; In response to determining at the queue that the received instruction is a secondary instruction that is dependent on one or more primary instructions associated with counters in the first subset of counters, stalling execution of the secondary instruction by the associated instruction pipeline if the counters in the first subset of counters associated with the primary instructions on which the secondary instruction depends indicate a hazard associated with the primary instructions; and In response to detecting by the monitor hardware logic that a hazard associated with a subject instruction has been resolved by an instruction pipeline of the plurality of instruction pipelines, a value of the counter associated with the subject instruction is adjusted to indicate that the hazard associated with the subject instruction has been resolved.

16. The method according to claim 15, wherein: Executing the computer executable instructions at the parallel processing unit further comprises: responsive to determining at the instruction decoder that the decoded instruction is a secondary instruction, determining whether each counter in a second subset of the counters associated with a primary instruction on which the secondary instruction depends indicates that a data hazard associated with the primary instruction has been resolved; forwarding the secondary instruction to one of the plurality of queues only in response to determining that each counter in a second subset of the counters associated with a primary instruction on which the secondary instruction depends indicates that the data hazard associated with the primary instruction has been resolved; and In response to determining that at least one counter in the second subset of counters associated with the primary instruction on which the secondary instruction depends indicates that the hazard associated with the primary instruction has not been resolved, the secondary instruction is descheduled until each counter in the second subset of counters associated with the primary instruction on which the secondary instruction depends indicates that the hazard associated with the primary instruction has been resolved.

17. A non-transitory computer-readable storage medium having stored thereon computer-readable instructions which, when executed at a computer system, cause the computer system to perform the method of any preceding claim.