Computational unit ordering for reducing divergence

Abstract

Techniques are described herein for reducing control flow divergence in single instruction multiple data processors. The method includes identifying control flow targets of different execution items at a point of diverging control flow, ordering the execution items based on the control flow targets, reorganizing the execution items based on the ordering, and executing with the reorganized execution items after the point of diverging control flow.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. non-provisional application No. 16 / 457,873, filed June 28, 2019, the contents of which are incorporated herein by reference. Background Technology

[0003] Single Instruction Multiple Data (“SIMD”) processors achieve parallelization of execution by using a single control flow module with multiple data items. When control flow depends on data, it can diverge because different execution threads may have different values ​​for the data on which the control flow depends. In such cases, different control flow paths are serialized, leading to slowdowns. Attached Figure Description

[0004] A more detailed understanding can be obtained from the following description, given by way of example in conjunction with the accompanying drawings:

[0005] Figure 1 This is a block diagram of an exemplary apparatus in which one or more features of this disclosure may be implemented;

[0006] Figure 2 The block diagram of the example device illustrates additional details related to performing processing tasks on the accelerated processing device.

[0007] Figure 3 This illustrates a ray tracing pipeline for rendering graphics using ray tracing techniques, based on an example.

[0008] Figure 4 The example illustrates serialization caused by divergent control flow;

[0009] Figure 5A The technique for reducing control flow divergence by recombining wavefront execution items across workgroups, as illustrated in the example, is shown.

[0010] Figure 5B This illustrates the control flow divergence that may occur without reorganization;

[0011] Figure 6A The technique for reducing control flow divergence by reorganizing execution items within the wavefront, as illustrated in the example, is shown.

[0012] Figure 6B This illustrates the control flow divergence that may occur without reorganization; and

[0013] Figure 7 This is a flowchart of a method for reorganizing execution items in APD, based on an example. Detailed Implementation

[0014] This document describes a technique for reducing control flow divergence in a single-instruction multiple-data processor. The method includes identifying control flow targets for different execution items at points where the control flow diverges, sorting the execution items based on the control flow targets, reorganizing the execution items based on the sorting, and executing the reorganized execution items after the points where the control flow diverges.

[0015] Figure 1 This is a block diagram of an exemplary device 100 that can implement one or more features of this disclosure. Device 100 includes, for example, a computer, gaming device, handheld device, set-top box, television, mobile phone, or tablet computer. Device 100 includes a processor 102, memory 104, storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 may also optionally include an input driver 112 and an output driver 114. It should be understood that device 100 includes... Figure 1 Additional components not shown.

[0016] In various alternatives, processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, where each processor core may be a CPU or a GPU. In various alternatives, memory 104 is located on the same die as processor 102, or is located separately from processor 102. Memory 104 includes volatile or non-volatile memory, such as random access memory (RAM), dynamic RAM, or cache.

[0017] Storage device 106 includes fixed or removable storage devices, such as hard disk drives, solid-state drives, optical disks, or flash drives. Input device 108 includes, but is not limited to, a keyboard, keypad, touchscreen, touchpad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connection (e.g., a wireless LAN card for transmitting and / or receiving wireless IEEE 802 signals). Output device 110 includes, but is not limited to, display device 118, speaker, printer, haptic feedback device, one or more lights, antenna, or network connection (e.g., a wireless LAN card for transmitting and / or receiving wireless IEEE 802 signals).

[0018] Input driver 112 communicates with processor 102 and input device 108, and allows processor 102 to receive input from input device 108. Output driver 114 communicates with processor 102 and output device 110, and allows processor 102 to send output to output device 110. Note that input driver 112 and output driver 114 are optional components, and device 100 will operate in the same manner in the absence of input driver 112 and output driver 114. Output driver 114 includes an accelerated processing unit (“APD”) 116 coupled to display device 118. APD 116 is configured to accept computation commands and graphics rendering commands from processor 102, process those computation commands and graphics rendering commands, and provide pixel output to display device 118 for display. As described in further detail below, APD 116 includes one or more parallel processing units configured to perform computations according to the Single Instruction Multiple Data (“SIMD”) paradigm. Therefore, although various functions are described herein as being performed by or in conjunction with APD 116, in various alternatives, the functions described as being performed by APD 116 are additionally or alternatively performed by other computing devices with similar capabilities, which are not driven by a main processor (e.g., processor 102) and are not configured to provide (graphics) output to display device 118. For example, it is conceivable that any processing system performing processing tasks according to the SIMD paradigm could be configured to perform the functions described herein.

[0019] Figure 2 The block diagram of the example device 100 illustrates additional details related to performing processing tasks on the APD 116. The processor 102 maintains one or more control logic modules in system memory 104 for execution by the processor 102. These control logic modules include an operating system 120, a driver 122, and an application program 126. These control logic modules control various features of the operation of the processor 102 and the APD 116. For example, the operating system 120 communicates directly with the hardware and provides an interface to the hardware for other software executing on the processor 102. The driver 122 controls the operation of the APD 116 by providing, for example, an application programming interface (“API”) to software (e.g., application program 126) that executes on the processor 102 to access various functions of the APD 116. In some implementations, the driver 122 includes a just-in-time (JIT) compiler that compiles programs for execution by processing units of the APD 116 (e.g., SIMD unit 138, discussed further in detail below). In other implementations, a just-in-time compiler is not used to compile the program, and a regular application compiler compiles the shader program for execution on the APD 116.

[0020] APD 116 executes commands and procedures for selected functions, such as graphics and non-graphics operations suitable for parallel and / or out-of-order processing. APD 116 is used to perform graphics pipeline operations, such as pixel operations, geometric calculations, and to render images to display device 118 based on commands received from processor 102. APD 116 also performs computational processing operations unrelated to or not directly related to graphics operations based on commands received from processor 102, such as operations related to video, physics simulations, computational fluid dynamics, or other tasks. APD 116 also performs computational processing operations related to ray-tracing based graphics rendering.

[0021] APD 116 includes a computation unit 132 comprising one or more SIMD units 138 that perform operations in parallel as requested by processor 102 according to a SIMD paradigm. A SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter, and thus execute the same program, but with different data. In one example, each SIMD unit 138 includes sixteen channels, each of which executes the same instructions simultaneously with other channels in the SIMD unit 138, but with different data. If not all channels need to execute a given instruction, assertions can be used to close channels. Assertions can also be used to execute programs with divergent control flow. More specifically, for programs with conditional branches or other instructions, where control flow is based on computation performed by a single channel, assertions of channels corresponding to currently unexecuted control flow paths and the serial execution of different control flow paths allow for arbitrary control flow. In one implementation, each of the computation units 132 may have a local L1 cache. In one implementation, multiple computation units 132 share an L2 cache.

[0022] The basic unit of execution in computation unit 132 is a work item. Each work item represents a single instance of a program that will be executed in parallel on a specific channel. Work items can be executed together as a “wavefront” on a single SIMD processing unit 138. The SIMD nature of the SIMD processing unit 138 means that multiple work items can be executed simultaneously in parallel. Work items executed together in this way on a single SIMD unit are part of the same wavefront. In some implementations or operating modes, the SIMD unit 138 executes the wavefront by executing each work item of the wavefront simultaneously. In other implementations or operating modes, the SIMD unit 138 executes different subsets of the work items in the wavefront in parallel. In one example, the wavefront comprises 64 work items, and the SIMD unit 138 has 16 channels (where each channel is a unit of hardware size designed to execute a single work item). In this example, the SIMD unit 138 executes the wavefront by executing 16 work items 4 times simultaneously.

[0023] One or more wavefronts are included in a “workgroup,” which comprises a set of work items designated to perform the same program. An application or other entity (“host”) requests a shader program to be executed by the accelerated processing device 116, specifying a “size” (the number of work items), and the command processor 136 generates one or more workgroups to perform the work. The number of workgroups, the number of wavefronts in each workgroup, and the number of work items in each wavefront are related to the size of the work requested by the host. In some implementations, the host may specify the number of work items in each workgroup for a specific request to perform the work, and this specification defines the number of workgroups generated by the command processor 136 to perform the work. As described above, the command processor 136 assigns the workgroups to one or more computing units 132, which execute an appropriate number of wavefronts to complete the workgroup.

[0024] The parallelism provided by computing unit 132 is suitable for graphics-related operations, such as pixel value calculation, vertex transformation, and other graphics operations. Therefore, in some instances, graphics pipeline 134, which receives graphics processing commands from processor 102, provides computational tasks to computing unit 132 for parallel execution.

[0025] In some implementations, the accelerated processing unit 116 performs ray tracing, a technique for rendering a 3D scene by intersecting simulated rays with objects in a test scene. Much of the work involved in ray tracing is performed by a programmable shader program, which executes on the SIMD unit 138 within the compute unit 132. Although some of the teachings presented herein are described in the context of performing ray tracing work on the APD 116, it should be understood that the various teachings presented herein can be applied to workloads other than ray tracing workloads.

[0026] Figure 3 A ray tracing pipeline 300 for rendering graphics using ray tracing technology, according to an example, is shown. The ray tracing pipeline 300 provides an overview of the operations and entities involved in rendering a scene using ray tracing. Ray generation shader 302, any-hit shader 306, most-hit shader 310, and miss shader 312 are stages of shader implementation that represent ray tracing pipeline stages, the functionality of which is performed by shader programs executing in SIMD unit 138. Any particular shader program at each particular shader implementation stage is defined by application-provided code (i.e., code provided by the application developer, pre-compiled by the application compiler and / or compiled by driver 122). Ray tracing stage 304 performs a ray intersection test to determine whether a ray hits a triangle. Ray tracing stage 304 can be performed by the shader program executing in SIMD unit 138 or by fixed-function hardware configured to perform ray intersection tests.

[0027] Various programmable shader stages (ray generation shader 302, any-hit shader 306, most-hit shader 310, and miss shader 312) are implemented as shader programs that execute on the SIMD unit 138. The command processor 136 coordinates the execution of the ray tracing pipeline 300. Specifically, the command processor 136 is a programmable unit that executes instructions to cause the various stages of the ray tracing pipeline 300 to execute on the APD 116. Additional details are provided elsewhere in this document.

[0028] The ray tracing pipeline 300 operates as follows: One or more computation units 132 execute a ray generation shader 302. The ray generation shader 302 requests the ray tracing stage 304 to execute one or more ray intersection tests. Each ray intersection test defines the origin and direction of the ray tracing operation, which determines whether the ray hits one or more triangles, or whether the ray does not hit any triangles.

[0029] Ray tracing stage 304 identifies one or more triangles that intersect with the ray to perform a ray intersection test, or, if no triangle intersects with the ray within a given distance, determines that the ray did not hit any triangle (i.e., the ray "missed"). Ray tracing stage 304 can be implemented in any technically feasible manner. In one example, ray tracing stage 304 is implemented as a shader program executing on one or more computing units 132. In another example, ray tracing stage 304 is implemented as fixed-function hardware.

[0030] In ray tracing phase 304, the execution of the most recently hit shader 310 is triggered for the triangle closest to the origin of the ray ray, or, if no triangle is hit, the miss shader is triggered. A typical use of the most recently hit shader 310 is to shade a material based on its texture. A typical use of the miss shader 312 is to shade a pixel with a color set by the skybox. It should be understood that the shader program defined for the most recently hit shader 310 and the miss shader 312 can implement a variety of techniques for shading pixels and / or performing other operations.

[0031] A typical way for the ray-generating shader 302 to generate rays is using a technique known as inverse ray tracing. In inverse ray tracing, the ray-generating shader 302 generates rays with an origin located at a point corresponding to the camera. The point where the ray intersects the plane defined to correspond to the screen defines a pixel on the screen whose color is determined using the ray. If the ray hits an object, the pixel is colored based on the most recently hit shader 310. If the ray does not hit an object, the pixel is colored based on the miss shader 312. Each pixel can project multiple rays, where the final color of the pixel is determined by some combination (or average) of the colors determined for each ray of the pixel. Any particular ray-generating shader 302 (or any other shader) can also be specified to execute any hit shader 306 for any hit between the ray and a triangle, even if such a hit is not the most recently hit.

[0032] The recently hit shader 310 and / or the recently missed shader 312 may generate their own rays, which enter the ray tracing pipeline 300 at a ray test point. These rays can be used for any purpose. A common use is to implement ambient lighting or reflections. In the example, when the recently hit shader 310 is invoked, it generates rays in various directions. For each object or light hit by the generated rays, the recently hit shader 310 adds illumination intensity and color at the hit location to the pixel corresponding to the recently hit shader 310 that generated the ray. It should be understood that although some examples of ways to render a scene using various components of the ray tracing pipeline 300 have been described, any of a wide variety of techniques can be used alternatively.

[0033] It should be understood that any shader program written for the most recently hit shader stage 310, the miss shader stage 312, or any hit shader stage 306 can perform any of the operations described elsewhere herein by the shader program written for the ray generation stage 302. For example, in addition to generating new rays to be provided to the ray test point for testing in the ray tracing stage 304, such a shader program can specify whether additional rays should be generated for tracing on a miss or a hit (starting again at the ray test point), what shader program to execute for any such additional pipeline, how to combine the color and / or brightness values ​​generated by such additional shader programs, and any other operations that can be performed by the ray generation shader 302.

[0034] The shader program is launched as a "kernel" on APD 116. The kernel specifies a particular shader program (e.g., a spliced ​​shader program) and a number of work items that will be executed as part of the kernel. Scheduler 136 divides the kernel into workgroups and assigns the workgroups to one or more compute units 132 for execution. The workgroups of the kernel begin execution, execute their instructions, and then terminate execution. The workgroups execute as one or more wavefronts within compute unit 132. The wavefronts execute as multiple work items executed concurrently on SIMD unit 138.

[0035] Each wavefront comprises work items that execute concurrently in a Single Instruction Multiple Data (“SIMD”) manner. More specifically, the SIMD unit 138 executes the shader program in a manner that uses a single instruction pointer to control the program execution of multiple work items, thus allowing multiple instructions to execute simultaneously. In the example, the four work items of the wavefront are executed on the SIMD unit. A portion of the execution control flow begins with the segment shown in the following pseudocode:

[0036]

[0037] Table 1: Exemplary SIMD Pseudocode

[0038] The first instruction adds the value in r2 to the value in r3 and stores the result in r1. The second instruction adds the values ​​in r1 through r5 and stores the result in r4. The third instruction stores the value in r4 at the address specified in r6. The wavefront channels executing this pseudocode execute simultaneously, so that multiple additions, multiplications, and stores are performed simultaneously for each channel. "r1" through "r6" represent register names.

[0039] Control flow can potentially diverge between channels in SIMD unit 138, and thus between work items on the wavefront. More specifically, some instructions modify the instruction pointer based on the value of a specific variable. In the example, a conditional branch is an instruction whose jump target is based on the evaluation result of a condition. In another example, a jump might target an address specified in a variable. When control flow diverges in this way, SIMD unit 138 serializes each of the possible paths to be executed by at least one channel. The following exemplary pseudocode illustrates a situation that could lead to control flow divergence.

[0040]

[0041] Table 2: Example SIMD divergence

[0042] In Table 2, each channel executes the instruction `add r1,r2,5`, which adds the value 5 to r2 and stores the result in r1. The `blz` instruction is a conditional branch; if the value in r1 is less than zero, the conditional branch branches. If the value in r1 is greater than zero, the control flow falls into segment 1, which includes some instructions followed by a jump instruction to the "Reconvert" label. Returning to the conditional branch, if the value in r1 is not greater than zero, the control flow continues to the label `LESS_THAN_ZERO` and executes segment 2. At the label `RECONVERGE`, the control flow reconverts.

[0043] If the first channel executing on the wavefront has a value of -10 stored in r2 when the first instruction is executed, then the register r1 of that channel will store a value of -5 after the first instruction, which will cause the first channel to execute the second segment at "LESS_THAN_ZERO". If the second channel executing on the same wavefront has a value of 1 in r2 when the first instruction is executed, then the register r1 of that channel will store a value of 6, which will cause the channel to not branch and execute the first segment. The execution of the first and second segments by different channels will be accomplished by executing each segment sequentially, where channels that do not execute a particular segment will be turned off. More specifically, the divergence will cause the SIMD unit 138 to execute the first segment for the second channel with the first channel turned off, and then execute the second segment for the first channel with the second channel turned off, which reduces processing efficiency because multiple channels that might have been executed simultaneously are now serialized.

[0044] Table 3 presents another example of divergent control flow.

[0045]

[0046] Table 3: Example SIMD Divergence - Function Pointers

[0047] In the example in Table 3, each channel performs ray tracing to detect triangle intersections. Each channel then identifies the material of the intersecting triangles and stores the address of the material shader for the identified material in register r1. Each channel then jumps to the address stored in r1. These addresses can be various material shaders shown (“MATERIAL_SHADER_1”, “MATERIAL_SHADER_2”, etc.). After executing the material shader, the channel jumps to “end_material_shaders”. If each channel hits a triangle using a different material shader, each of these material shaders will be serialized, resulting in a deceleration equal to the number of wavefront channels—this would represent complete deparallelization. It should be noted that the material shader is a code segment used for ray tracing, and executing this segment provides color for rays that intersect with a triangle (e.g., in the most recently hit shader phase) or miss a triangle and are therefore colored by the skybox (e.g., in the miss phase).

[0048] In some examples, on a SIMD processor, the point where a branch instruction exists is called a "branch point." A branch at a branch point points to one or more branch targets. A branch with a fixed target has a single branch target, while a branch with a variable target may have more than one. There are also convergence points, which are points where diverging paths due to different branching will necessarily converge. The sequence of instructions that begins at a branch target is referred to herein as an "adopted path." The sequence of instructions that begins with the instruction immediately following a conditional branch ("unadopted point") is referred to herein as an "unadopted path." In this document, adopted paths and unadopted paths are collectively referred to as "code paths." Each code path extends from a branch target or unadopted point to a convergence point or branch point. Essentially, each code path defines a sequence of instructions in which the combination of channels executing the code path cannot change (this change can occur due to branching or convergence).

[0049] To execute a sequence of instructions containing branches, the SIMD processor evaluates the branch instructions for each channel and, based on the results, sets bit values ​​within an execution bitmask for each code path reachable from the branch. Each bit in the bitmask is associated with a channel on the wavefront being executed. One bit value in the bitmask (e.g., "1") indicates that the corresponding channel will execute the code path. Another bit value in the bitmask (e.g., "0") indicates that the corresponding channel will not execute the code path.

[0050] After determining bitmasks for different code paths, the SIMD processor advances or modifies the instruction pointer as needed until all code paths to be executed by at least one channel have actually been executed. For code paths whose bitmasks indicate that no channel executes the code path, the SIMD processor modifies the instruction pointer to skip that code path. Generally, modifying the instruction pointer as needed involves changing the address of the last instruction of one code path executed by at least one channel to the address of another code path executed by at least one channel. If the two code paths to be executed are sequentially arranged in memory, this modification may include simply incrementing the instruction pointer, or this modification may involve a "true branch," meaning that the instruction pointer is modified in a way that differs from simply incrementing it by setting the instruction pointer to the address of the first instruction of the code path to be executed.

[0051] Figure 4 The diagram illustrates serialization caused by divergent control flow, based on an example. Table 400 shows several code segments, listed as "Aggregate Segment," "F1," "F2," "F3," and "F4." In the diagram, time progresses downwards. The marker "O" in the given box indicates that the channel executes the code segment within a given time period. In the aggregate segment, it is assumed that each of channels 1 through 4 executes the segment together. Therefore, each channel in the aggregate segment has a marker "O." As a result of the aggregate segment, it is determined that channel 1 will execute F1 without executing F2, F3, or F4; channel 2 will execute F2 without executing F1, F3, or F4; channel 3 will execute F3 without executing F1, F2, or F4; and channel 4 will execute F4 without executing F1, F2, or F3. It can be seen that each of F1, F2, F3, and F4 executes in a different time period, thus the execution of the wavefronts including channels 1 through 4 is deparallelized. More specifically, in the first time period, with channels 2 through 4 off, channel 1 executes function F1. In the second time period, with channels 1 and 3 to 4 closed, channel 2 executes function F2. In the third time period, with channels 1 to 2 and 4 closed, channel 3 executes function F3. In the fourth time period, with channels 1 to 3 closed, channel 4 executes function F4.

[0052] To reduce control flow divergence, computation unit 132 reorganizes execution items at control flow divergence points. In some implementations, the term "execution item" refers to a work item. In other implementations, the term "execution item" refers to an execution thread that is more granular than a work item. More specifically, multiple logical execution threads can be executed within a single work item by sequentially executing such multiple logical threads. In such examples, each of the multiple logical execution threads is an execution item. Two techniques for reorganizing execution items at divergence points include the technique of computation unit 132 reorganizing execution items across different wavefronts of a work group and the technique of computation unit 132 reorganizing execution items within a wavefront.

[0053] Figure 5A This illustrates a technique for reducing control flow divergence by reorganizing execution items across workgroups, based on an example. In this example, there is a one-to-one correspondence between work items and execution items—each work item executes one execution item. Figure 5A In the scenario, a workgroup 502 includes two wavefronts—wavefront 1 504(1) and wavefront 2 504(2). Workgroup 502 is a collection of work items executed together in a single computing unit 132. The work items of workgroup 502 are executed together as wavefronts. Wavefronts include work items executed simultaneously on the SIMD unit 138, such as those related to... Figure 4 As described. All work items of a wavefront can be executed simultaneously in SIMD unit 138, but a wavefront may also include a number of work items greater than the number of data channels in SIMD unit 138. Typically, such a wavefront will include a number of work items equal to an integer multiple of the number of data channels in SIMD unit 138. Execution of such a wavefront will be performed by sequentially executing a subset of the wavefront's work items. In the example, the wavefront includes 64 work items and SIMD unit 138 includes 16 data channels. In this example, the wavefront is executed by executing work items 1 to 16, then 17 to 32, then 33 to 48, and then 49 to 64.

[0054] Different wavefronts within a single workgroup are not executed in the simultaneous SIMD manner described herein, although such wavefronts can be executed simultaneously on different SIMD units 138 of a single computing unit 132. However, a characteristic of a workgroup is that computing unit 132 supports synchronization between different workgroups. "Synchronization" refers to the wavefront's ability to execute barriers that halt the execution of all wavefronts involved in the barrier until a certain condition is met. Wavefronts within a workgroup also have the ability to communicate via local memory within computing unit 132 during execution.

[0055] exist Figure 5A Table 500 illustrates the execution of workgroup 502. Workgroup 502 includes two wavefronts 504. Wavefront 1 504(1) includes work items 1 through 4, while wavefront 2 504(2) includes work items 5 through 8. Instruction pointers indicate the code segment executed by each wavefront 504 at a given point in time. Time progresses from top to bottom. Certain entries in the table correspond to code sections that a particular work item may or may not execute. These sections include functions F1 through F4 executed by one or more work items. Aggregate sections are executed by all work items of wavefront 504. An “O” mark indicates that a particular work item executes one of these code sections, and a blank rectangle indicates that a particular work item does not execute one of these code sections.

[0056] In the aggregation section, each work item determines which of the functions F1 through F4 it will execute. Note that the term "function" refers to a portion of the shader program being executed by work group 502. A barrier and reorganization phase is executed after the aggregation section. The barrier and reorganization phase halts the execution of each wavefront 504 until the barrier and reorganization phase has been completed for each wavefront 504 undergoing reorganization. Figure 5A In the example, these wavefronts 504 include wavefront 1 504 (1) and wavefront 2 504 (2). The barrier and reorganization phase reorganizes work items across different wavefronts 504 based on the result of the aggregation portion indicating which of the functions F1 to F4 the work item will execute. Typically, the goal of reorganization is to reduce wavefront divergence by exchanging work items between wavefronts (in the example, higher divergence is associated with a higher total number of functions to be executed). Generally, divergence reduction is achieved by reducing the number of control flow targets of at least one wavefront.

[0057] In the example shown, the work items in both wavefront 1 504(1) and wavefront 2 504(2) execute all functions F1 through F4. This is considered a 4-fold divergence because the computational unit 132 executing those work items must serialize each of functions F1 through F4. By grouping work items branching to the same control flow target together, the reorganization reduces the total number of functions executed on the wavefront. The example reorganization results in wavefront 1 504(1) having work items that specify the execution of functions F1 and F2 without executing F3 and F4, and wavefront 2 504(2) having work items that specify the execution of functions F3 and F4 without executing F1 and F2. Specifically, wavefront 1 504(1) includes work items 1, 2, 5, and 6, each of which is specified to execute either F1 or F2 without executing F3 or F4, and wavefront 2 504(2) includes work items 3, 4, 7, and 8, each of which is specified to execute either F3 or F4 without executing F1 or F2. Through this reorganization, each wavefront 504 diverges by only two times. It should be noted that the code executed by each wavefront 504 still includes functions that are not executed by a particular work item, but this code is skipped, resulting in little or no execution time dedicated to those functions for wavefront 504s with work items that do not execute those functions.

[0058] Although a specific exemplary sequence of instructions is shown, wherein this specific sequence includes determining a function to be performed for each work item, and the work item performing said function, it should be understood that regarding Figure 5AThe described techniques can be applied to any instruction sequence that leads to divergent control flow. In any case, the barrier and reorganization phase will examine the total number of divergent control flow targets in workgroup 502 and attempt to assign as few divergent control flow targets as possible to each wavefront 504. Control flow targets can be identified by the instruction pointer address targeted by the branch instruction, by the decision of whether conditional branching is used, or by any other technically feasible method. Assigning as few divergent control flow targets as possible to each wavefront 504 can be achieved by sorting the wavefronts based on targets, dividing the sorted list according to the number of work items in each wavefront 504, and assigning the divided work items to different wavefronts 504. Moving a work item from one wavefront 504 to another wavefront can be accomplished by any technically feasible method, such as by modifying the list of work items assigned to each wavefront 504, and by copying execution status data, such as register values ​​and flag values, from the location where the work item was previously executed to the location where the work item will be moved.

[0059] The barrier and reorganization phases can be implemented entirely in software, for example, by instructions inserted by a runtime or offline compiler, or by special hardware support. In the example, either or both of the destination-based sorting and the reorganization sorting by execution item can be implemented by fixed-function hardware triggered by special instructions executed by each wavefront 504 of workgroup 502.

[0060] Figure 5B It was shown that in the absence Figure 5A The execution that would occur under the reorganization scenario is illustrated. As mentioned above, each wavefront 502 includes work items for executing functions F1 to F4. Therefore, the execution of these functions would be 4-fold divergent without reorganization, since each function is executed sequentially.

[0061] Figure 6A This illustrates a technique for reducing control flow divergence by reorganizing execution items within a wavefront, based on an example. In the example shown, each job executes two execution items. More specifically, the shader program executed by the wavefront is configured such that two instances of a particular workload—two execution items—execute sequentially. Each execution item, executed as part of a particular job, is executed in a specific time slot. It should be noted that multiple different jobs can, of course, execute concurrently. Thus, multiple jobs can execute their time slot 1 concurrently, and then their time slot 2 concurrently. In the ray tracing example, each time slot can correspond to a different ray. In the example, the shader program determines the triangle where the rays intersect. The shader program executes a material shader based on the material of the intersecting triangle. Since each time slot of a job corresponds to a different ray, different material shaders can be executed in different time slots of a single job.

[0062] The sequential performance of the two execution items allows them to be swapped between time slots to reduce divergence. Figure 6A The example illustrates this type of exchange. In the example, each of the four channels of wavefront 602 executes two items. Channel 1 executes items 1 and 2, channel 2 executes items 3 and 4, channel 3 executes items 5 and 6, and channel 4 executes items 7 and 8. An aggregation section is performed on each channel to identify which of the functions F1, F2, F3, and F4 should be executed for each item. Although not shown, this aggregation section can be performed once for each item.

[0063] The results of the aggregation section are shown: Execution item 1 will execute function F1, execution item 2 will execute function F2, execution item 3 will execute function F2, execution item 4 will execute function F3, execution item 5 will execute function F1, execution item 6 will execute function F4, execution item 7 will execute function F3, and execution item 8 will execute function F4. A total of three functions will be executed in time slot 1 and a total of three functions will be executed in time slot 2. This means that the divergence factor will be 3 in each time slot. In the reorganization phase, computation unit 132 reorganizes the execution items across different channels to reduce divergence. Specifically, the reorganization phase sorts the execution items according to their objectives, divides the sorted execution items, and assigns the divided, sorted execution items to time slots. In the example shown, the sorting objectives are F1 (item 1), F1 (item 5), F2 (item 2), F2 (item 3), F3 (item 4), F3 (item 7), F4 (item 6), and F4 (item 8). During the reorganization phase, the execution items for functions F1 and F3 are assigned to time slot one, and the execution items for functions F2 and F4 are assigned to time slot two. Time slot one executes, then time slot two executes, each with two functions—a divergence factor of 2. Specifically, functions F1 through F4 are executed sequentially, as shown in the figure. This sequence occurs twice—once per time slot. In time slot one, items 1 and 5 execute function F1 simultaneously, and items 4 and 7 execute function F3 simultaneously. In time slot one, functions F2 and F4 are skipped because no channel executes these functions. In time slot two, items 2 and 3 execute F2 simultaneously, and items 6 and 8 execute function F4 simultaneously, where functions F1 and F3 are skipped.

[0064] Although specific exemplary instruction sequences are shown, wavefront techniques are typically performed as follows: The compiler generates shader programs with two or more time slots, each a copy of a specific workload to be executed for different execution items. At points where control flow diverges, the compiler inserts reorganization code to reorganize the execution items across time slots. The reorganization code categorizes the execution items into groups based on control flow destination. The reorganization code attempts to assign as few groups as possible to each time slot. This reorganization reduces divergence in at least one time slot, thereby reducing the overall execution time.

[0065] The reorganization phase can be implemented via software (e.g., by instructions inserted into the shader program by a runtime or offline compiler) or via at least partial hardware support. In the example, the shader program executes special instructions that cause fixed-function hardware to reorganize execution items across time slots as described.

[0066] It should be noted that although the functions in a time slot appear to be executed sequentially within each time slot, each function may actually be executed repeatedly for each time slot. Within a time slot, time slot 1 executes function 1, then time slot 2 executes function 1, then time slot 1 executes function 2, time slot 2 executes function 2, and so on. The teachings of this paper are applicable to any technique that performs multiple iterations of a specific segment of a diverging control flow. In various examples, the runtime or offline compiler transforms the shader program to perform multiple iterations of the segments of the diverging control flow as described.

[0067] Figure 6B It was shown that in the absence Figure 6A This will occur during the reorganization process. Specifically, because there are three channels performing different functions in each time slot, each time slot will execute with a 3x divergence, which is consistent with... Figure 6A The execution that occurs in the middle is slower compared to the speed reduction.

[0068] Figure 7 This is a flowchart of method 700 for reorganizing execution items in APD 116, based on an example. Although regarding... Figure 1 The system described up to Figure 6 has been described, but it should be understood that any system configured to perform the steps of method 700 in any technically feasible order falls within the scope of this disclosure.

[0069] Method 700 begins at step 702, where computation unit 132 identifies control flow targets for different execution items at points in the diverging control flow. As described elsewhere herein, two techniques for recombining execution items are disclosed. In a first technique, execution items are recombined between time slots within a single wavefront. This first technique may be referred to herein as the “intra-wavefront technique.” In a second technique, execution items are recombined across wavefronts of a wavegroup. This second technique may be referred to herein as the “inter-wavefront technique.” In either technique, the executing shader program reaches a point in the diverging control flow where different execution items execute different portions of the shader program due to having different branch targets (e.g., due to different conditional results or due to different address targets of variable target jump instructions). In step 702, these different branch targets are identified for different execution items. In some examples, the branch target for a particular execution item includes the address of the execution item’s destination. In other examples, the branch target includes an indication of whether a conditional branch is employed (i.e., whether a condition is met).

[0070] In the example, each executor performs a ray-tracing operation to identify the material of the triangles intersecting with the rays. At this point, different executors encounter different triangles with different materials, thus executing different material shaders. It should be noted that the different material shaders are inlined into the currently executing shader program; therefore, executing a specific material shader is done by jumping to the appropriate inline function, rather than by stopping the execution of the current shader and starting a new material shader. In other words, at least a portion of the ray tracing pipeline is implemented by a single "monolithic" shader program that includes shaders for different stages of the ray tracing pipeline, as well as instructions for controlling the execution flow between different stages and for moving data between different stages.

[0071] In step 704, calculation unit 132 sorts the execution items based on the control flow objective. In the example of divergence based on jumps to a variable objective, the control flow objective is the target of the jump. In the example of divergence based on conditional branching, the control flow objective is the result of the condition. Sorting based on the control flow objective involves sorting the execution items such that execution items with the same control flow objective are grouped together. The result of the sorting is data indicating the sorted order of the execution items. In the example, execution items 1 to 8 exist and execute conditional branches. Execution items 1, 3, 4, and 7 use branches, while execution items 2, 5, 6, and 8 do not use branches. The data indicating the sorted order indicates that execution items 1, 3, 4, and 7 are in the first group, and execution items 2, 5, 6, and 8 are in the second group.

[0072] In step 706, computation unit 132 reorganizes execution items based on a sorting operation. In the intrawavelength technique of reorganizing execution items across time slots, reorganizing execution items involves assigning each group of sorted execution items to a specific time slot. In the example above, execution items 1, 3, 4, and 7 will be assigned to the first time slot, and execution items 2, 5, 6, and 8 will be assigned to the second time slot. As described elsewhere herein, a time slot is an iteration of a sequentially repeating portion of a shader program. More specifically, in order to reorganize execution items within a wavefront, at least some portions of the shader program are configured to execute in two or more iterations, where each iteration corresponds to a different time slot. Since divergence is related to the number of different code segments executed at a given time, having multiple time slots allows for the reorganization of execution items between time slots to execute more execution items with the same control flow objective within the same time slot. Of course, the number of execution items in each sorted group may differ from the number of execution items in the wavefront. However, even in the case of assigning two or more groups to specific time slots, sorting reduces divergence, thereby reducing the total execution time.

[0073] In some implementations, the compiler introduces slot construction into the shader program. One example of a compiler is, for instance, a runtime compiler that executes as part of a driver 122. Another example is an offline compiler that executes as part of an application compiler that compiles the application that triggers the execution of the shader program. The compiler introduces slot construction by causing at least a portion of the shader program to execute two or more iterations—one iteration per slot. In each iteration, a different executor executes a portion of the shader program. Furthermore, since the purpose of slots is to allow for reduction of control flow divergence caused by different executors taking different control flow paths, each iteration can select any possible divergence path (even if each executor might only take one such path). In the example, if there are four different code segments that can be executed in a given portion of the shader program, where variable jumps are directed to one of the possible code segments, then in each slot, the executor can execute any possible code segment. Diverging control flow can still occur within a given slot, and this is achieved by serializing each of the different possible control flow paths in the SIMD unit 138, which is common for diverging control flow.

[0074] In the inter-wavefront technique, the reorganization of execution items occurs across wavefronts. More specifically, the sorting data described above is still used to group execution items. However, instead of reorganizing execution items across time slots within a single wavefront, groups are assigned together to specific wavefronts. In the example above where each wavefront is four work items wide, execution items 1, 3, 4, and 7 will be assigned to the first wavefront, and execution items 2, 5, 6, and 8 will be assigned to the second wavefront. Similar to the intra-wavefront technique, multiple groups can be assigned to each wavefront, but this still reduces divergence. In step 708, the shader program executes with reorganized execution items after passing the point of divergent control flow.

[0075] It should be understood that many variations are possible based on the disclosure herein. While features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements. In one example, although these techniques have been described in the context of a SIMD processor, they can also be used in a Multiple Instruction Multiple Data (“MIMD”) processor. In another example, while the ordering is stated as being based on the code segments to be executed, it can also be based on other criteria, such as the texture a channel is acquiring or the direction of the light associated with the channel. Ordering based on such criteria is not directly related to ordering based on which code segments the channel will execute. However, such ordering is more likely to result in channels with similar execution flows being grouped together. In the example, channels with similar light directions are likely to hit the same triangle and therefore execute the same material shader. Channels acquiring the same texture are also likely to execute similar instruction sequences, as acquiring the same texture may instruct the channel to execute the same shader (e.g., a material shader).

[0076] The various functional units shown in the figures and / or described herein (including, but not limited to, processor 102, input driver 112, input device 108, output driver 114, output device 110, accelerated processing device 116, scheduler 136, graphics processing pipeline 134, computing unit 132, SIMD unit 138, ray tracing pipeline 300, ray generation shader 302, ray tracing stage 304, any hit shader 306, miss shader 312, or most recently hit shader 310) can be implemented as a general-purpose computer, processor, or processor core, or as a program, software, or firmware stored on a non-transitory computer-readable medium or other medium and executable by a general-purpose computer, processor, or processor core. The provided methods can be implemented in a general-purpose computer, processor, or processor core. For example, suitable processors include general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and / or state machines. Such processors can be manufactured by configuring the manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data (including netlists) (which can be stored on a computer-readable medium). The result of such processing can be a mask, which is then used in the semiconductor manufacturing process to manufacture processors that implement various aspects of the implementation scheme.

[0077] The methods or flowcharts provided herein can be implemented in a computer program, software, or firmware incorporated into a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROMs and DVDs).

Claims

1. A method for reducing control flow divergence in a single-instruction multiple-data processor, the method comprising: Control flow objectives for different execution items of a wavefront are identified at the point of divergent control flow, wherein the wavefront is divided into multiple time slots for execution. The execution items are sorted based on the identified control flow objectives to generate a sorted group of execution items; The execution items are reorganized by exchanging the wavefront execution items between the plurality of time slots based on the sorted execution item groups, such that within the time slots of the plurality of time slots, more execution items have the same control flow objective compared to the case without the reorganization; as well as Based on the reorganized execution item, the wavefront is executed after the point of the divergent control flow.

2. The method of claim 1, wherein identifying the control flow objective of the different execution items includes identifying whether the different execution items employ branching.

3. The method of claim 1, wherein identifying the control flow target of the different execution items includes identifying the target address of the jump.

4. The method of claim 1, wherein sorting the execution items includes grouping execution items having the same control flow objective together into the execution item group.

5. The method of claim 1, further comprising: The compiler adds instructions to the shader program to implement the sorting and reorganization.

6. An apparatus for reducing control flow divergence in a single-instruction multiple-data processor, the apparatus comprising: The memory is configured to store shader programs; as well as The single instruction multiple data processor is configured to execute the shader program using multiple execution items, and is configured to: At the point of divergent control flow, the control flow objectives of different execution items among the plurality of execution items of the wavefront are identified, wherein the wavefront is divided into multiple time slots for execution. The execution items are sorted based on the identified control flow objectives to generate a sorted group of execution items; The execution items are reorganized by exchanging the wavefront execution items between the plurality of time slots based on the sorted execution item groups, such that within the time slots of the plurality of time slots, more execution items have the same control flow objective compared to the case without the reorganization; as well as Based on the reorganized execution item, the wavefront is executed after the point of the divergent control flow.

7. The apparatus of claim 6, wherein identifying the control flow objective of the different execution items includes identifying whether the different execution items employ branching.

8. The apparatus of claim 6, wherein identifying the control flow target of the different execution items includes identifying the target address of the jump.

9. The apparatus of claim 6, wherein sorting the execution items includes grouping execution items having the same control flow objective together into the execution item group.

10. A non-transitory computer-readable medium storing instructions, which, when executed by a processor, cause the processor to reduce the divergence of control flow in a single-instruction multiple-data processor by: Control flow objectives for different execution items of a wavefront are identified at the point of divergent control flow, wherein the wavefront is divided into multiple time slots for execution. The execution items are sorted based on the identified control flow objectives to generate a sorted group of execution items; The execution items are reorganized by exchanging the wavefront execution items between the plurality of time slots based on the sorted execution item groups, such that within the time slots of the plurality of time slots, more execution items have the same control flow objective compared to the case without the reorganization, wherein execution items of the same group are reorganized into the same time slot; as well as Based on the reorganized execution item, the wavefront is executed after the point of the divergent control flow.

11. The non-transitory computer-readable medium of claim 10, wherein identifying the control flow objective of the different execution items includes identifying whether the different execution items employ branching.

12. The non-transitory computer-readable medium of claim 10, wherein identifying the control flow target of the different execution items includes identifying the target address of the jump.

Images