Efficient execution of atomic instructions for single instruction multi-thread (SIMT) architecture

Abstract

The invention relates to efficient execution of atomic instructions for a single instruction multithreading (SIMT) architecture. A first set of threads having the same address corresponding to the shared memory is identified from a group of active threads associated with an instruction for updating the shared memory. A first thread in the first set of threads is selected. Instructions are executed for the first thread using the same address for accessing the shared memory. Attempts to the remaining threads in the first set of threads execute the instruction are delayed until after the first thread has been executed and until at least one of the remaining threads in the first set of threads is not ensured to fail in executing the instruction.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to parallel processing architectures, and more particularly, to improved techniques for efficient execution of atomic instructions in a single instruction multiple thread (SIMT) processing architecture. Background Art

[0002] Many computer applications can be accelerated by using parallel processing techniques, for example, where the same instruction can be executed in parallel on multiple data elements. For example, in image and media processing applications, the processing of a large number of pixels, image blocks, and / or vertex sets can be mapped to different computing threads or processing channels that can be executed in parallel. For example, in a (SIMT) processing architecture, a common instruction (or instruction stream) can be executed in parallel using a group of processing threads. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present disclosure will be more fully understood through the detailed description given below and the accompanying drawings of various embodiments of the present disclosure. However, the accompanying drawings should not be considered to limit the present disclosure to specific embodiments, but are only for explanation and understanding.

[0004] Figure 1 An example computing environment is shown in accordance with at least one embodiment.

[0005] Figure 2 A flow diagram is shown of an example method for executing an atomic compare and store (CAST) instruction using a peel loop, in accordance with at least one embodiment.

[0006] Figure 3 A flow chart illustrating an example method for efficiently executing atomic instructions of a SIMT architecture, according to at least one embodiment.

[0007] Figure 4 A computer system in accordance with at least one embodiment is shown.

[0008] Figure 5 A parallel processing unit ("PPU") is shown in accordance with at least one embodiment.

[0009] Figure 6 A general processing cluster ("GPC") is shown in accordance with at least one embodiment.

[0010] Figure 7 A memory partitioning unit of a parallel processing unit ("PPU") is shown in accordance with at least one embodiment.

[0011] Figure 8 A streaming multiprocessor in accordance with at least one embodiment is shown.

[0012] Figure 9 A computer system in accordance with at least one embodiment is shown.

[0013] Figure 10A A parallel processor in accordance with at least one embodiment is shown.

[0014] Figure 10B A partition unit according to at least one embodiment is shown.

[0015] Figure 10C A processing cluster is shown in accordance with at least one embodiment.

[0016] Figure 10D A graphics multiprocessor in accordance with at least one embodiment is shown. DETAILED DESCRIPTION

[0017] The systems and methods described herein may be used for non-autonomous vehicles, semi-autonomous vehicles (e.g., in one or more adaptive driver assistance systems (ADAS)), manned and unmanned robots or robotic platforms, warehouse vehicles, off-road vehicles, vehicles connected to one or more trailers, flying boats, boats, space shuttles, emergency response vehicles, motorcycles, electric or motorized bicycles, aircraft, construction vehicles, underwater vehicles, drones, and / or other types of vehicles. Further, the systems and methods described herein may be used for various purposes, such as, but not limited to, for applications related to machine control, machine motion, machine actuation, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twins, data center processing, conversational AI, light transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation of 3D assets, cloud computing, and / or any other suitable application.

[0018] The disclosed embodiments may be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aerial systems, medical systems, boating systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems including one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least in part in a data center, systems for performing conversational AI operations, systems for hosting live streaming applications, systems for presenting one or more of virtual reality content, augmented reality content, or mixed reality content, systems for performing light transport simulations, systems for performing collaborative content creation of 3D assets, systems implemented at least in part using cloud computing resources, and / or other types of systems.

[0019] To improve performance, parallel processors (e.g., parallel processing units (PPUs) and graphics processing units (GPUs)) typically employ processing techniques such as pipelining to attempt to process as much data as possible in parallel across different parts of the parallel processing pipeline. Parallel processors may implement a single instruction multiple thread (SIMT) processing architecture, which is designed to maximize the amount of parallel processing within the parallel processing pipeline. In a SIMT processing architecture, groups of parallel threads can execute program instructions together synchronously as frequently as possible to improve processing efficiency.

[0020] At a high level, in a SIMT processing architecture, a common instruction (or instruction stream) can be executed in parallel using a set of processing threads. At the hardware level, parallel execution of multiple threads is performed using parallel execution units, such as SIMT execution units (e.g., special purpose function units, texture units, load / store units, etc.). For example, the parallel execution units may be able to simultaneously perform a variety of different computational operations (e.g., integer and floating-point arithmetic operations, comparison operations, Boolean operations, etc.).

[0021] In practice, it is common for multiple threads to execute the same instruction concurrently, attempting to concurrently access and modify memory that is shared between the multiple threads. Therefore, some SIMT processing architectures can implement atomic operations to ensure that certain operations on shared memory locations are performed atomically without interference from other threads. This is particularly important in situations where multiple threads may attempt to read and modify the same memory location at the same time, to prevent data corruption and ensure consistency. For example, atomic operations can allow exclusive access to read, modify, and write data, where a memory location is locked to prevent simultaneous access by different threads. This lock can then be removed at the end of the read-modify-write operation, where these operations allow serialization of execution while stopping other threads.

[0022] Simple atomic operations (e.g., integer addition, subtraction, etc.) can be accelerated to single instructions using enhanced memory hardware to execute these simple atomic instructions and serialize threads accessing the same location in shared memory. However, more complex operations (e.g., floating-point arithmetic, arbitrary calculations, etc.) can be performed by dedicated functional units outside of the memory. Therefore, a software process can use a peel loop to execute (serially as needed) threads in a group (e.g., a warp). For example, in a read-modify-write (RMW) operation (e.g., compare and store (CAST), compare and swap (CAS), test and set (TAS), etc.) that can be executed on multiple threads and can update shared memory, a peel loop can be operated to divide a group of threads into thread groups associated with the same logical unit of shared memory. The shared memory can be divided into logical units (also referred to herein as "memory banks" or "memory bodies") to manage complexity and improve performance. The peel loop may further operate to select one of the threads in the set of threads associated with the same memory bank to proceed with the operation and preemptively terminate (also referred to herein as "fast-fail" or "atomic-fail") the other threads, preventing the other threads included in the set from executing the remaining instructions of the operation until the selected thread executes its operation. Thus, the peel loop may prevent the remaining threads from continuously attempting (and failing) to execute. In this case, fast-failure of operations for threads associated with the same memory bank but at different shared memory addresses may unnecessarily increase power consumption and resource utilization.

[0023] In some cases, conventional implementations of executing threads in a group using a peel loop may unnecessarily increase the computational workload, resulting in poor performance. For example, a conventional system may use the aforementioned peel loop to perform a CAST operation on a selected set of threads in the group. The CAST operation may include at least three parameters: a memory location (e.g., an address in shared memory), an expected value, and a new value. Each thread in the group may read the memory location and compare the value stored at the location with the expected value. If the values ​​do not match, the thread does not write to the memory location and may return an integer (e.g., "0") indicating failure. If the values ​​match, the thread writes the new value to the memory location and may return an integer (e.g., "1") indicating success. In some cases, if two or more threads attempt to access the same bank in memory, a peel loop can quickly cause all but one thread to fail, as described above. Conventional implementations of such peel loops may include instructions that convert the returned integer ("0" or "1") to a Boolean value ("false" or "true," respectively) before branching to the start of the peel loop. Converting the returned integer value to the corresponding Boolean value is an instruction that is unnecessarily added to the peel loop, increasing the computational workload of the peel loop and resulting in suboptimal performance.

[0024] In some conventional implementations, for each iteration of a peel loop, each thread in a set of threads accessing the same address in shared memory may attempt to access data at the same address. In one iteration of the peel loop, one thread may succeed, while the remaining threads in the set may fail (fast fail, failed compare, etc.). Subsequent threads may update the data stored at the same address in shared memory with a new value. In subsequent iterations of the peel loop, the remaining threads may access the new value, now stored at the same address, and perform their operations. This redundant access to shared memory increases the latency of each iteration of the peel loop, thereby reducing overall performance.

[0025] The embodiments of the present disclosure address the above-mentioned limitations and / or other limitations of the existing SIMT architecture by optimizing (e.g., by a compiler, software, etc.) the peel loop to simplify (e.g., reduce the number of necessary iterations, or eliminate unnecessary operations within the loop) the peel loop. In operations that can be performed on behalf of multiple threads, the peel loop can divide a group of threads into thread sets (e.g., sub-groups) that target the same memory location, and use the peel loop to jointly perform the operations of each thread set. In an iteration of the peel loop, one of the threads in the thread set that targets the same memory location (memory address) can be selected to continue operating on the shared memory location and cause the remaining threads in the thread group of the set to fail atomically / quickly (e.g., terminate before the comparison sequence). By causing the concurrently executed thread set that targets the same location in the shared memory to fail atomically, rather than causing the thread set that targets the same memory group of the shared memory to fail atomically, the total number of false failures can be reduced, thereby reducing the total number of peel loop iterations and improving overall performance.

[0026] In some embodiments, the peel loop can be further optimized by returning a Boolean value to each thread in a corresponding predicate register, indicating whether the thread successfully performed its operation. By returning the status directly to the predicate register, the overall performance of the peel loop can be improved because the atomic post-transformation sequence can be avoided.

[0027] As described above, subsequent threads in the peel loop can update the data stored at the address in shared memory with the new value, which can then be retrieved by other threads in subsequent iterations of the peel loop. In some embodiments, the peel loop can be further optimized by returning the new value to the private registers corresponding to the remaining threads. In subsequent iterations of the peel loop, the remaining threads can use the value from the corresponding register without first loading the value from shared memory into the corresponding private register. By avoiding loading values ​​from shared memory into private registers, shared memory accesses can be reduced, thereby improving overall performance.

[0028] Figure 1 1 is a block diagram illustrating a computing system 100 according to at least one embodiment. In some embodiments, the computing system 100 may be a heterogeneous computing system that includes one or more types of computing units, such as one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs), one or more data processing units (DPUs), one or more field programmable gate arrays (FPGAs), and / or one or more application-specific integrated circuits (ASICs). As shown, for example, the computing system 100 may include a general-purpose processor 110 (e.g., a multi-core CPU) and a parallel processor 120 (e.g., a general-purpose GPU (GPGPU)).

[0029] The general-purpose processor 110 may be designed for fast serial processing of program instructions, while the parallel processor 120 may be designed for highly parallel processing of program instructions (e.g., computational instructions). The parallel processor 120 may operate as a coprocessor for the general-purpose processor 110, wherein portions of a computer application (e.g., data-parallel, computationally intensive portions of the application) are offloaded to the parallel processor 120 for execution.

[0030] As an illustrative example, computing system 100 may be used to execute a computer application 140. Computer application 140 may include a set of program instructions that may include a mix of sequential instruction portions 141 that may be executed as a series of one or more threads 145 on general-purpose processor 110 and parallel instruction portions 142 that may be executed in parallel as multiple threads 146 on parallel processor 120. For example, a portion of a computer application may contain programming instructions that are executed multiple times but independently on potentially different data, which may be executed as multiple threads 146 on parallel processor 120. The threads 146 may be organized into one or more thread blocks 147 (e.g., as an array or grid 148 of thread blocks 147), which may be executed concurrently by parallel processor 120. In some embodiments, parallel processor 120 may include one or more multiprocessors 130, with one or more thread blocks 147 distributed to each multiprocessor 130 for execution. Individual threads in thread block 147 may be executed concurrently by multiprocessor 130 , and multiprocessor 130 may execute multiple thread blocks concurrently.

[0031] In some embodiments, multiprocessor 130 may employ a SIMT (single instruction, multiple thread) architecture to concurrently execute multiple threads. For example, multiprocessor 130 may be configured to create, manage, schedule, and execute threads in groups of parallel threads, which may be referred to as warps. When multiprocessor 130 is given one or more thread blocks to execute, it may partition them into separate thread groups that can be independently scheduled for execution.

[0032] In some embodiments, the multiprocessor 130 can be configured to execute one common instruction in a group of threads at a time (e.g., a warp, a half warp, a quarter warp, etc.). Therefore, when all threads in a group agree on their execution path, full efficiency can be achieved. The individual threads in a group of threads can be started together at the same program address (e.g., a common instruction in an instruction sequence), but can be assigned their own instruction address counter (or program counter) and register state, allowing each thread to branch and execute independently. If the individual threads diverge through conditional control structures (e.g., conditional branches, conditional function calls, or conditional returns), different branch paths (e.g., the results of the divergence) can be executed serially. When (e.g., all) branch paths are executed, the threads can converge back to the same execution path. In some cases, program instructions can provide synchronization points at which all threads in the group converge (e.g., some threads can wait for all threads in the group to arrive). Threads in a group of threads participating in the current instruction can be referred to as active threads, while threads not on the current instruction can be referred to as inactive (or disabled) threads.

[0033] In some embodiments, the multiprocessor 130 may include a function execution unit that can be configured to perform various operations, including, for example, integer and floating-point arithmetic operations (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, XOR), shift operations, random number generation operations, and other computational operations (e.g., planar interpolation and trigonometric functions, exponential functions, and logarithmic functions). In some embodiments, the multiprocessor 130 may include one or more execution units 131. The execution unit 131 may be configured to execute a single instruction on multiple data sets. In some embodiments, the execution unit 131 is capable of executing a common instruction for each thread in a group of threads (e.g., using different sets of source operands and generating different sets of result operands for each thread).

[0034] In some embodiments, multiprocessor 130 may include one or more register files or register sets for use by functional execution units of multiprocessor 130. In some embodiments, for example, multiprocessor 130 may include a private register set 133 that provides temporary storage for operands of a data path connected to execution unit 131. In some embodiments, private registers 133 may be partitioned and allocated to individual threads in a set of threads executed by multiprocessor 130, with the allocated portions serving as private register space for each of the individual threads. In some embodiments, private registers 133 may be statically partitioned (e.g., having a fixed size for each individual thread) and dynamically allocated for use by the individual threads. In some embodiments, multiprocessor 130 may include a shared register set that is accessed by some or all threads in a set of threads executed by multiprocessor 130.

[0035] In some embodiments, the multiprocessor 130 may include one or more additional register sets. In some embodiments, for example, the multiprocessor 130 may include a special register set that may store predefined, platform-specific information, such as thread parameters (e.g., thread identifiers (within thread blocks), lane identifiers (within warps), warp identifiers, block identifiers, etc.), clock counters, and / or performance monitoring information. In some embodiments, the multiprocessor 130 may include a predicate register set 135 that may be used to store predicates (e.g., 1-bit Boolean values) that may be used to support instruction predication (e.g., conditional branch predication). In some embodiments, for example, an instruction may accept an optional predicate guard operand that, if determined to be true, causes the instruction to be executed, and, if determined to be false, prevents the instruction from being executed.

[0036] In some embodiments, multiprocessor 130 may include shared memory 136. Shared memory 136 may be allocated (statically or dynamically) and accessed by some or all of the threads in the set of threads executed by multiprocessor 130. In some embodiments, multiprocessor 130 may also have access to a global memory space, such as global memory space on device memory 121 of parallel processor 120, which may be provided to some or all of the threads in the set of threads executed by multiprocessor 130.

[0037] In some embodiments, computing system 100 may include software compiler logic 150 that may be used to compile a computer application (e.g., computer application 140) from program code that may be stored in memory 112. For example, software compiler logic 150 may be used to compile program source code 151 into binary code 154 that may be executed by general-purpose processor 110 and / or parallel processor 120. Program source code 151 may include a mixture of code, some of which may be designed to execute on general-purpose processor 110 ("host code") and some of which may be designed to execute on parallel processor 120 ("device code"). In some embodiments, software compiler logic 150 may be operable to separate device code from host code and compile the code separately. For example, software compiler logic 150 may compile device source code into one or more function kernels and then modify the host code to include the necessary runtime function calls to load and launch each compiled function kernel. Software compiler logic 150 may then compile the modified host code to obtain binary code 154 that may be executed by computing system 100 on general-purpose processor 110 and parallel processor 120.

[0038] In some embodiments, the software compiler logic 150 can compile the program source code 151 or a portion thereof (e.g., the device code in the program source code 151) in multiple stages, generating one or more intermediate code sets (e.g., intermediate assembly code 152 and low-level assembly code 153) before finally obtaining the binary code 154. For example, the program source code 151 can be written in a high-level programming language (e.g., C, C++, Java, Python, Fortran, DirectCompute, OpenACC, etc.). The software compiler logic 150 can compile the program source code 151 written in a high-level programming language into an intermediate assembly code 152 (e.g., PTX code, Khronos SPIR code, LLVM IR code, etc.), which can use a specific instruction set architecture (ISA). In some embodiments, the intermediate assembly code 152 can use an instruction set suitable for general parallel programming, which can be designed to be efficiently executed by the parallel processor 120. In some embodiments, the software compiler logic 150 can compile the device code in the program source code 151 into an intermediate assembly code 152 designed to be architecture-independent, so that the same code can be used for different parallel processor architectures. In some embodiments, the computer application (or portion thereof) may be written directly into intermediate assembly code 152. In some embodiments, software compiler logic 150 may be operable to compile (e.g., further compile) the intermediate assembly code into low-level assembly code 153 (e.g., source code and assembly (SASS) code). Low-level assembly code 153 may use another ISA (e.g., different from intermediate assembly code 152), which may be a native architecture that uses target architecture instructions for a particular parallel processor architecture. In some embodiments, software compiler logic 150 may use low-level assembly code 153 to generate binary microcode for native execution on parallel processor 120.

[0039] In some embodiments, the software compiler logic 150 may be operable to generate (e.g., optimized) program code (e.g., intermediate assembly code 152, low-level assembly code 153, or binary code 154), which may, for example, improve execution efficiency and resource utilization. For example, the software compiler logic 150 may seek to optimize the program code to expose sufficient parallelism, coalesce memory accesses, ensure consistent execution within a thread group, etc., which may improve execution of the program code on the parallel processors 120 (and multiprocessors 130).

[0040] In some embodiments, for example, the software compiler logic 150 may be operable to perform branch predication (or control flow flattening) to ensure convergent execution of multiple threads (e.g., all threads in a thread group). In some embodiments, for example, the software compiler logic 150 may analyze program code and determine instances where branch predication can be used to more efficiently implement the code. For example, the software compiler logic 150 may determine that a branch created in the program code by a program loop (e.g., a for, while, or do-while loop) or a logic block (e.g., an if or switch block) may be cheaper than predicating and executing all threads (e.g., cheaper than branching the code and executing each branch path serially) and may expand or flatten these loops or logic blocks. That is, instructions whose execution depends on conditional control structures (e.g., conditional branches, conditional function calls, or conditional returns) are not skipped, but are associated with a condition code or predicate for each thread, which is set to true or false based on the control condition. Although these instructions may be scheduled for execution by all threads, only those instructions (or threads) with true predicate values ​​are actually executed. For an instruction (or thread) with a false predicate value, the address may not be evaluated, the operands may not be read, and / or the result may not be written. It may be cheaper to predicate and execute the instruction for all threads.

[0041] In some cases, multiple threads may concurrently attempt to execute the same atomic update instruction (e.g., a read-modify-write atomic instruction) on the same location in shared memory 136. During an atomic update instruction on the same memory location in shared memory 136, only one thread in a set of threads accessing the same memory location may succeed, while the remaining threads in the set may fail. In this case, each thread (e.g., using execution unit 131) makes redundant failed attempts to execute the atomic update instruction, unnecessarily increasing power consumption and resource utilization. Therefore, in some embodiments, the software compiler logic 150 may generate a peel loop for any atomic read-modify-write instruction (RMW) (e.g., compare and store (CAST), compare and swap (CAS), etc.) that is to be executed by a group of threads targeting shared memory 136. In at least one embodiment, the software compiler logic 150 may generate a peel loop for any RMW operation in shared memory 136 for which corresponding hardware is lacking to perform the RMW operation.

[0042] For example, the software compiler logic 150 can determine that the atomic CAST is to be performed by a group of active threads. In this case, the software compiler logic 150 can generate a peel loop that can be used to partition the thread group into one or more thread sets that target the same location in shared memory 136 and execute the threads in a particular thread set in a serial manner (e.g., for each thread in one or more thread sets in the active thread group) using the execution unit 131.

[0043] As an illustrative example, software compiler logic 150 may generate the following peel loop for the above-described CAST operation:

[0044] LD R1,[R0] / / 10 Atom stripping loop:

[0045] FADD R2,R1,R3 / / 20

[0046] ATOMS.CAST.SPIN P0,R1,[R0],R1,R2 / / 30

[0047] @! P0 BRA atomic peel loop / / 40 The load (LD) instruction may operate to read data stored at a location in the shared memory 136 identified based on a memory address of the shared memory 136. Specifically, [R0] may refer to a memory address from which the data is to be read. The LD instruction may copy the data to register R1 (e.g., to a private register of a particular thread). The LD instruction may be executed by each thread in a particular thread set to perform a CAST operation using the data stored at address [R0] in the shared memory 136.

[0048] The peel loop may begin with a floating point add (FADD) instruction in which two integer values ​​are added together to be executed for each thread in the thread set. In some embodiments, the FADD assembly instruction may use the parallel execution unit 131 (e.g., using a dedicated functional unit of the execution unit 131) to execute the add FADD instruction separately for each thread. Specifically, the FADD instruction adds the values ​​of source operands R1 and R3, which may be a private register 133 (e.g., in the register space of each thread) and places the result in a destination operand R2, which may be a private register 133 (e.g., in the register space of each thread). The result in the destination operand R2 may be used as a new value in the CAST instruction at step 30 described below. In at least one embodiment, the value of the source operand R3 may be a unique value for each thread in the set.

[0049] The peel loop may proceed to execute an atomic shared memory compare and store (ATOMS.CAST) instruction, wherein a value stored at a memory location 136 is compared to a source value, and if the two values ​​match, a new value may be stored at the memory location. In at least one embodiment, the ATOMS.CAST instruction is operable to execute ATOMS.CAST for each thread serially using the parallel execution unit 131, such as a dedicated functional unit of the parallel execution unit 131. Specifically, the ATOMS.CAST instruction is operable to compare the value of source operand R1 (referred to herein as the "original value" or "expected value") with the value stored at address [R0] in shared memory. If the values ​​match (the CAST succeeds), the instruction is operable to update the location in shared memory corresponding to address [R0] with the value of source operand R2 (referred to herein as the "new value").

[0050] In at least one embodiment, if the ATOMS.CAST instruction succeeds (the values ​​match), the ATOMS.CAST instruction can operate to return the new value stored at source operand R2 to each register R1 (e.g., a private register of a particular thread) of other threads in the peel loop (e.g., other threads attempting to execute CAST instructions at the same location in memory). If the ATOMS.CAST instruction fails (the values ​​do not match), the ATOMS.CAST instruction can operate to return the value stored at address [R0] to each register R1 (e.g., a private register of a particular thread). By returning the new value directly to the private thread registers, the LD instruction can be placed outside the peel as shown above, thereby avoiding the delay associated with repeated accesses to shared memory for each iteration of the peel loop.

[0051] In at least one embodiment, the ATOMS.CAST instruction may also return a Boolean value indicating success or failure. For example, in response to determining that the value of source operand R1 matches the value stored at address [R0] in shared memory, the ATOMS.CAST instruction may operate to set predicate P0 to true for the corresponding thread. In response to determining that the value of source operand R1 does not match the value stored at address [R0] in shared memory, the ATOMS.CAST instruction may operate to set predicate P0 to false for the corresponding thread. By returning the Boolean value indicating success or failure directly to the predicate register of the corresponding thread, the integer-to-Boolean conversion step may be avoided, thereby improving the overall performance of the peel loop.

[0052] At this point, the peel loop may proceed to the next iteration using those threads for which the ATOMS.CAST instruction is still pending (e.g., if predicate P0 is false, returning to the Atomic Peeling Loop), ultimately terminating when no threads remain pending (no threads for which predicate P0 is false). In some embodiments, for example, instruction 40 may accept a predicate guard operand and, if the predicate guard operand is determined to be true, may cause instruction 40 to be executed (branching to the Atomic Peeling Loop), and, if the predicate guard operand is determined to be false, may prevent execution of the branch instruction.

[0053] In some embodiments, the peel loop can be further optimized, for example, using an optional feature of the shared memory atomic operation, referred to herein as ".SPIN," to reduce the number of iterations in the peel loop. In the example shown above, the .SPIN feature can be specified as a suffix to the ATOMS.CAST opcode. The ATOMS.CAST.SPIN instruction can operate to select (e.g., based on a thread identifier) ​​a single thread to attempt the ATOMS.CAST instruction and cause the remaining threads that target the same shared memory location to fail quickly. In at least one embodiment, to cause the remaining threads to fail quickly, the ATOMS.CAST.SPIN instruction can operate to set the predicate P0 to false for each remaining thread (e.g., a thread not selected for execution) before attempting the CAST instruction. Thus, a thread that could fail a CAST comparison sequence (e.g., due to another thread updating a shared memory location) can fail the CAST comparison sequence before attempting the CAST comparison sequence. Furthermore, by modifying ATOMS.CAST.SPIN to cause threads accessing the same shared memory address (rather than the same memory bank) to fail quickly, each iteration of the peel loop can complete more useful work because comparisons that could fail are avoided, thereby improving overall performance. In contrast, each iteration of the peel loop wastes fewer cycles.

[0054] It should be understood that ATOMS.CAST is used herein by way of example and not limitation, and it should be noted that the software compiler logic 150 may generate peel loops for other read-modify-write instructions that atomically update shared memory. For example, the software compiler logic 150 may generate peel loops using the techniques described herein for instructions such as Fetch and Add (FAS), Compare and Swap (CAS), Test and Set (TAS), and the like.

[0055] In some cases, multiple threads may concurrently attempt to execute the same atomic update instruction (e.g., a read-modify-write atomic instruction) on the same memory bank at different addresses of shared memory 136, thereby generating a cross-thread port conflict. In at least one embodiment, hardware (e.g., a memory access queue, a pipeline, etc.) may be implemented to serialize such atomic update instructions within a single iteration of a peel loop.

[0056] Figure 2 and Figure 3 Example methods according to embodiments of the present disclosure are shown. For simplicity and clarity, these methods are described as a series of operations. However, according to the present disclosure, these operations can be performed in other orders and / or simultaneously, and can be performed together with other operations not presented or described herein. In addition, when implementing the methods according to the present disclosure, not all of the operations shown may be required. Those skilled in the art will also understand and appreciate that these methods can be represented as a series of interrelated states or events through state diagrams. In addition, it should be understood that the disclosed methods can be stored on an article of manufacture. The term "article of manufacture" as used herein is intended to cover a computer-readable device or storage medium equipped with a computer program and / or executable instructions that, when executed, affect one or more operations.

[0057] Figure 2 A flow chart is shown of an example method 200 for executing an atomic compare and store (CAST) instruction on an execution unit using a peel loop, in accordance with at least one embodiment of the present disclosure.

[0058] In the SIMT programming and execution model, atomic update (e.g., ATOMS.CAST) instructions can be executed concurrently by multiple threads, where each thread may update a different location in shared memory. However, typically, multiple threads concurrently execute the same instruction at the same location in shared memory. In this case, a peel loop can partition a group of threads into sets (e.g., subgroups) that exhibit memory address consistency for source operand values, where the operations of the respective sets are performed collectively using the peel loop.

[0059] At operation 210 , processing logic may identify a set of threads from the active thread group to execute the same instruction at the same address in shared memory, such as Figure 1 shared memory 136.

[0060] At operation 220, processing logic may copy the data stored at the same address to the data register of each thread in the identified set. For example, processing logic may copy the data stored at the same address in shared memory 136 to a private register of the plurality of private registers 133 of each thread in the identified set.

[0061] At operation 230, processing logic may begin peeling the loop by selecting a thread in the thread set to continue executing instructions. For example, processing logic may select a thread to continue based on a thread identifier.

[0062] At operation 240, processing logic may execute the instruction for the selected thread. For example, processing logic may use execution unit 131 to execute the ATOMS.CAST.SPIN instruction for the selected thread. In at least one embodiment, operations 240, 250, and 260 may be performed in a single instruction. Specifically, at operation 240, a compare and update (e.g., CAST) may be performed by comparing an expected value to a value stored at the same memory address. In response to a determination that the values ​​match, processing logic may update the shared memory at the memory address (e.g., the destination operand) with the new value specified by the source operand of the instruction.

[0063] At operation 250 , processing logic may copy the result to a private register (eg, private register 133 ) of each thread in the set of threads.

[0064] At operation 260, processing logic may update a predicate register, such as predicate register 135, for each thread in the set. For example, if the corresponding thread successfully executes the instruction, processing logic may set the predicate register to true. If the corresponding register fails to execute the instruction, processing logic may set the predicate register to false.

[0065] At block 262, processing logic may set the predicate (eg, predicate register 135) for the selected thread to true.

[0066] At block 264, processing logic may set a predicate (e.g., predicate register 135) to false for each remaining thread in the set. In some embodiments, this may be referred to as "fast failing" the remaining threads because the predicates for the remaining threads are set to false before the remaining threads attempt to execute the instruction, indicating a failure to execute the instruction.

[0067] At operation 260, processing logic may determine whether any threads are still waiting to execute. If so, processing logic may return to operation 230; otherwise, the loop may terminate and processing logic may continue to execute the next instruction. In some embodiments, for example, processing logic may accept a predicate guard operand and, if the predicate guard operand is determined to be true, may cause a branch to the beginning of the peel loop; if the predicate guard operand is determined to be false, execution may continue to the next update instruction.

[0068] It should be understood that Figure 2The peel loop described herein is used as an example only and not as a limitation. Based on the techniques described herein, processing logic can be operated to generate peel loops for other atomic read-modify-write instructions. For example, processing logic can generate a peel loop for executing a CAS instruction, a TAS instruction, etc.

[0069] Figure 3 A flow chart of an example method 300 for efficiently executing atomic instructions of a SIMT architecture is shown. At operation 302 of method 300, processing logic may identify a first set of threads from active thread groups associated with instructions that update shared memory that have the same address corresponding to shared memory.

[0070] At operation 304 of method 300 , processing logic may select a first thread from the first set of threads.

[0071] At operation 306 of method 300, processing logic may execute the instruction of the first thread using the same address to access the shared memory. In at least one embodiment, in response to executing the instruction of the first thread, processing logic may store a Boolean value in a predicate register corresponding to the first thread. The Boolean value indicates that the first thread successfully executed the instruction.

[0072] At operation 308 of method 300, processing logic may delay attempting to execute the instruction on the remaining threads in the first thread set until after the first thread is executed and until it is no longer certain that at least one of the remaining threads in the first thread set has failed to execute the instruction. In at least one embodiment, to delay execution of the instruction by the remaining threads in the first thread set, processing logic may store a Boolean value in one or more predicate registers corresponding to the remaining threads. The Boolean value indicates that the remaining threads have failed to execute the instruction.

[0073] In at least one embodiment, the instruction is a compare and store (CAST) instruction. To execute the CAST instruction, processing logic compares a first value stored at the same memory address in the shared memory with an expected value. In response to determining that the first value matches the expected value, processing logic may write a second value to the shared memory at the same memory address. In at least one embodiment, processing logic may further write the second value to one or more private registers corresponding to one or more threads of the first set. In at least one embodiment, after executing the first thread, processing logic may use the second value stored in the corresponding private register of the one or more private registers to execute the CAST instruction of a second thread in the remaining threads of the first thread set using the second value stored in the corresponding private register of the one or more private registers.

[0074] In at least one embodiment, the shared memory includes multiple logical units. In at least one embodiment, the processing logic may further execute instructions from threads of an active thread group having different addresses corresponding to the same logical unit in the multiple logical units of the shared memory. For example, in some cases, multiple threads in the active thread group may concurrently attempt to execute the same instruction on the same memory bank at different addresses of the shared memory, thereby generating a cross-thread port conflict. In at least one embodiment, the processing logic may implement hardware (e.g., memory access queues, pipelines, etc.) to serialize the execution of these instructions.

[0075] Figure 4 A computer system 400 is shown in accordance with at least one embodiment. In at least one embodiment, the computer system 400 is configured to implement the various processes and methods described throughout this disclosure.

[0076] In at least one embodiment, computer system 400 includes at least one central processing unit ("CPU") 402 connected to a communication bus 410 implemented using any suitable protocol, such as PCI ("Peripheral Component Interconnect"), Peripheral Component Interconnect Express ("PCI-Express"), AGP ("Accelerated Graphics Port"), HyperTransport, or any other bus or point-to-point communication protocol. In at least one embodiment, computer system 400 includes main memory 404, which may take the form of random access memory ("RAM"). Control logic (e.g., implemented as hardware, software, or a combination thereof) and data are stored in main memory 404. In at least one embodiment, a network interface subsystem ("network interface") 422 provides an interface to other computing devices and networks for receiving data from other systems and sending data to other systems using computer system 400.

[0077] In at least one embodiment, computer system 400 includes one or more input devices 408, a parallel processing system 412, and one or more display devices 406. Display devices can be implemented using conventional cathode ray tubes ("CRTs"), liquid crystal displays ("LCDs"), light emitting diode ("LED") displays, plasma displays, or other suitable display technologies. In at least one embodiment, user input is received from input devices 408, such as a keyboard, mouse, touchpad, microphone, etc. In at least one embodiment, each of the modules described herein can be located on a single semiconductor platform to form a processing system.

[0078] Software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate may be provided in Figure 4 A system is used to perform software compilation operations in an embodiment of the present invention, including generating and optimizing peel loops to execute atomic memory update instructions on execution units.

[0079] In at least one embodiment, a computer program in the form of machine-readable executable code or computer control logic algorithms is stored in the main memory 404 and / or the secondary storage device. According to at least one embodiment, if the computer program is executed by one or more processors, it enables the system 400 to perform various functions. In at least one embodiment, the memory 404, the storage device, and / or any other storage device are possible examples of computer-readable media. In at least one embodiment, the secondary storage device can refer to any suitable storage device or system, such as a hard drive and / or a removable storage drive, representing a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk ("DVD") drive, a recording device, a universal serial bus ("USB") flash memory, etc.

[0080] In at least one embodiment, the architecture and / or functionality of the various previous figures are implemented in the context of a CPU 402, a parallel processing system 412, an integrated circuit capable of implementing at least a portion of the capabilities of the CPU 402 and the parallel processing system 412, a chipset (e.g., an integrated circuit designed to operate and be sold as a unit to perform the relevant functions, etc.), and / or any suitable combination of integrated circuits. In at least one embodiment, the architecture and / or functionality of the various previous figures are implemented in the context of a general-purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, a dedicated system, etc. In at least one embodiment, the computer system 400 can take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless handheld device), a personal digital assistant ("PDA"), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, a mobile telephone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic device.

[0081] In at least one embodiment, the parallel processing system 412 includes a plurality of parallel processing units ("PPUs") 414 and associated memory 416. In at least one embodiment, the PPUs 414 are connected to a host processor or other peripheral devices via an interconnect 418 and a switch 420 or multiplexer. In at least one embodiment, the parallel processing system 412 distributes computational tasks among parallelizable PPUs 414, for example, as part of distributing computational tasks across multiple graphics processing unit ("GPU") thread blocks. In at least one embodiment, memory is shared and accessible (e.g., for read and / or write access) among some or all of the PPUs 414, although such shared memory may result in a performance penalty relative to using local memory and registers resident on the PPUs 414. In at least one embodiment, the operation of the PPUs 414 is synchronized using a command such as _syncthreads(), where all threads in a block (e.g., executing on multiple PPUs 414) reach a certain execution point in the code before continuing.

[0082] Figure 5 A parallel processing unit ("PPU") 500 is shown in accordance with at least one embodiment. In at least one embodiment, PPU 500 is configured with machine-readable code that, if executed by PPU 500, causes PPU 500 to perform some or all of the processes and techniques described throughout this disclosure. In at least one embodiment, PPU 500 is a multi-threaded processor implemented on one or more integrated circuit devices and utilizes multithreading as a latency hiding technique designed to process computer-readable instructions (also referred to as machine-readable instructions or simply instructions) on multiple threads in parallel. In at least one embodiment, a thread refers to an execution thread and is an instantiation of an instruction set configured to be executed by PPU 500. In at least one embodiment, PPU 500 is a graphics processing unit ("GPU") configured to implement a graphics rendering pipeline for processing three-dimensional ("3D") graphics data to generate two-dimensional ("2D") image data for display on a display device, such as a liquid crystal display ("LCD") device. In at least one embodiment, PPU 500 is configured to perform computations such as linear algebra operations and machine learning operations. Figure 5 The example parallel processor is shown for purposes of illustration only and should be construed as a non-limiting example of a processor architecture contemplated within the scope of the present disclosure, and any suitable processor may be employed in addition to and / or in place of the processor.

[0083] In at least one embodiment, one or more PPUs 500 are configured to accelerate high-performance computing ("HPC"), data center, and machine learning applications. In at least one embodiment, the PPU 500 is configured to accelerate deep learning systems and applications, including the following non-limiting examples: autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analysis, molecular simulation, drug discovery, disease diagnosis, weather forecasting, big data analysis, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations.

[0084] In at least one embodiment, the PPU 500 includes an input / output ("I / O") unit 506, a front-end unit 510, a scheduler unit 512, a work distribution unit 514, a hub 516, a crossbar switch ("XBar") 520, one or more general processing clusters ("GPCs") 518, and one or more partitioning units ("memory partitioning units") 522. In at least one embodiment, the PPU 500 is connected to a host processor or other PPUs 500 via one or more high-speed GPU interconnects ("GPU interconnects") 508. In at least one embodiment, the PPU 500 is connected to a host processor or other peripheral devices via a system bus 502. In at least one embodiment, the PPU 500 is connected to local memory including one or more memory devices ("memory") 504. In at least one embodiment, the memory devices 504 include one or more dynamic random access memory ("DRAM") devices. In at least one embodiment, one or more DRAM devices are configured and / or configurable as a high-bandwidth memory ("HBM") subsystem, where multiple DRAM dies are stacked within each device.

[0085] In at least one embodiment, the high-speed GPU interconnect 508 may refer to a wire-based, multi-lane communication link used by a system to extend and include one or more PPUs 500 in combination with one or more central processing units ("CPUs"), supporting cache coherency between the PPU 500 and the CPU, and CPU mastering. In at least one embodiment, data and / or commands are transmitted by the high-speed GPU interconnect 508 to / from other units of the PPU 500, such as one or more copy engines, video encoders, video decoders, power management units, and the like, through the hub 516. Figure 5 Other components that may not be explicitly shown.

[0086] In at least one embodiment, the I / O unit 506 is configured to receive data from the host processor ( Figure 5506 sends and receives communications (e.g., commands, data) via the system bus 502. In at least one embodiment, the I / O unit 506 communicates with the host processor directly via the system bus 502, or through one or more intermediate devices (such as a memory bridge). In at least one embodiment, the I / O unit 506 can communicate with one or more other processors (such as one or more PPUs 500) via the system bus 502. In at least one embodiment, the I / O unit 506 implements a Peripheral Component Interconnect Express ("PCIe") interface for communicating over the PCIe bus. In at least one embodiment, the I / O unit 506 implements an interface for communicating with external devices.

[0087] In at least one embodiment, the I / O unit 506 decodes data packets received via the system bus 502. In at least one embodiment, at least some of the packets represent commands configured to cause the PPU 500 to perform various operations. In at least one embodiment, the I / O unit 506 sends the decoded commands to various other units of the PPU 500 as specified by the commands. In at least one embodiment, the commands are transmitted to the front end unit 510 and / or to the hub 516 or other units of the PPU 500, such as one or more replication engines, video encoders, video decoders, power management units, etc. ( Figure 5 In at least one embodiment, I / O unit 506 is configured to route communications between and among the various logical units of PPU 500.

[0088] In at least one embodiment, a program executed by a host processor encodes a command stream in a buffer that provides a workload to the PPU 500 for processing. In at least one embodiment, the workload can include many instructions and data to be processed by those instructions. In at least one embodiment, the buffer is an area of ​​memory that is accessible (e.g., read / write) by both the host processor and the PPU 500—the host interface unit can be configured to access this buffer in system memory connected to the system bus 502 via memory requests transmitted on the system bus 502 via the I / O unit 506. In at least one embodiment, the host processor writes a command stream into the buffer and then sends a pointer to the beginning of the command stream to the PPU 500, so that the front end unit 510 receives pointers to one or more command streams and manages the one or more command streams, reading commands from the command streams and forwarding the commands to the various units of the PPU 500.

[0089] In at least one embodiment, the front end unit 510 is coupled to a scheduler unit 512, which configures the various GPCs 518 to process tasks defined by one or more command streams. In at least one embodiment, the scheduler unit 512 is configured to track state information associated with the various tasks managed by the scheduler unit 512, where the state information may indicate which GPC 518 the task is assigned to, whether the task is active or inactive, a priority associated with the task, etc. In at least one embodiment, the scheduler unit 512 manages the execution of multiple tasks on one or more GPCs 518.

[0090] In at least one embodiment, the scheduler unit 512 is coupled to a work distribution unit 514, which is configured to dispatch tasks for execution on the GPCs 518. In at least one embodiment, the work distribution unit 514 tracks a number of scheduled tasks received from the scheduler unit 512, and the work distribution unit 512 manages a pending task pool and an active task pool for each GPC 518. In at least one embodiment, the pending task pool includes a number of time slots (e.g., 32 time slots) containing tasks assigned to be processed by a particular GPC 518, and the active task pool may include a number of time slots (e.g., 4 time slots) for tasks being actively processed by the GPC 518. When a task completes execution in a GPC 518, the task is evicted from the active task pool of the GPC 518, and one of the other tasks from the pending task pool is selected and scheduled for execution on the GPC 518. In at least one embodiment, if an active task on a GPC 518 has become idle, such as while waiting for a data dependency to be resolved, the active task is evicted from the GPC 518 and returned to the pending task pool, while another task in the pending task pool is selected and scheduled for execution on the GPC 518.

[0091] In at least one embodiment, work distribution unit 514 communicates with one or more GPCs 518 via XBar (crossbar) 520. In at least one embodiment, XBar 520 is an interconnect network that couples many units of PPU 500 to other units of PPU 500 and can be configured to couple work distribution unit 514 to a specific GPC 518. In at least one embodiment, one or more other units of PPU 500 can also be connected to XBar 520 via hub 516.

[0092] In at least one embodiment, tasks are managed by a scheduler unit 512 and dispatched to one of the GPCs 518 by a work distribution unit 514. In at least one embodiment, a GPC 518 is configured to process tasks and generate results. In at least one embodiment, the results can be consumed by other tasks within the GPC 518, routed to a different GPC 518 via an XBar 520, or stored in memory 504. In at least one embodiment, the results can be written to memory 504 via a partition unit 522, which implements a memory interface for reading and writing data to memory 504. In at least one embodiment, the results can be sent to another PPU 500 or CPU via a high-speed GPU interconnect 508. In at least one embodiment, a PPU 500 includes a number U of partition units 522, which is equal to the number of independent and distinct memory devices 504 coupled to the PPU 500.

[0093] In at least one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications to be executed on the host processor to schedule operations for execution on the PPU 500. In at least one embodiment, multiple computing applications are executed simultaneously by the PPU 500, and the PPU 500 provides isolation, quality of service ("QoS"), and independent address spaces for the multiple computing applications. In at least one embodiment, the application generates instructions (e.g., in the form of API calls) that cause the driver kernel to generate one or more tasks for execution by the PPU 500, and the driver kernel outputs the tasks to one or more streams processed by the PPU 500. In at least one embodiment, each task includes one or more related thread groups, referred to herein as warps. In at least one embodiment, a warp includes multiple related thread groups (e.g., 32 threads) that can execute in parallel. In at least one embodiment, cooperating threads can refer to multiple threads that include instructions to execute tasks and can exchange data through shared memory.

[0094] The software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate may be provided in Figure 5 A system is used to perform software compilation operations in an embodiment of the present invention, including generating and optimizing peel loops to execute atomic memory update instructions on execution units.

[0095] Figure 6A general processing cluster ("GPC") 600 is shown in accordance with at least one embodiment. In at least one embodiment, the GPC 600 is Figure 5 518. In at least one embodiment, each GPC 600 includes multiple hardware units for processing tasks, and each GPC 600 includes a pipeline manager 602, a pre-raster operations unit ("preROP") 604, a raster engine 608, a work distribution crossbar ("WDX") 616, a memory management unit ("MMU") 618, one or more data processing clusters ("DPCs") 606, and any suitable combination of components.

[0096] In at least one embodiment, the operation of GPC 600 is controlled by pipeline manager 602. In at least one embodiment, pipeline manager 602 manages the configuration of one or more DPCs 606 for processing tasks assigned to GPC 600. In at least one embodiment, pipeline manager 602 configures at least one of one or more DPCs 606 to implement at least a portion of a graphics rendering pipeline. In at least one embodiment, DPC 606 is configured to execute vertex shading programs on a programmable streaming multiprocessor ("SM") 614. In at least one embodiment, pipeline manager 602 is configured to route packets received from work distribution units to appropriate logic units within GPC 600. In at least one embodiment, some packets may be routed to fixed-function hardware units within preROP 604 and / or raster engine 608, while other packets may be routed to DPC 606 for processing by primitive engine 612 or SM 614. In at least one embodiment, pipeline manager 602 configures at least one of DPC 606 to implement a neural network model and / or compute pipeline.

[0097] In at least one embodiment, the preROP unit 604 is configured to route data generated by the raster engine 608 and the DPC 606 to the raster operations ("ROP") unit in the partition unit 522, as described above in conjunction with Figure 5Described in more detail. In at least one embodiment, the preROP unit 604 is configured to perform optimizations for color blending, organize pixel data, perform address translation, and the like. In at least one embodiment, the raster engine 608 includes several fixed-function hardware units configured to perform various raster operations, and the raster engine 608 also includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, a tile aggregation engine, and any suitable combination thereof. In at least one embodiment, the setup engine receives transformed vertices and generates plane equations associated with geometric primitives defined by the vertices; the plane equations are sent to the coarse raster engine to generate coverage information for the primitives (e.g., x, y coverage masks for tiles); the output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and to the clipping engine, where fragments outside the viewing frustum are clipped. In at least one embodiment, those fragments remaining after clipping and culling can be passed to the fine raster engine to generate attributes for the pixel fragments based on the plane equations generated by the setup engine. In at least one embodiment, the output of raster engine 608 includes fragments to be processed by any suitable entity, such as fragments processed by a fragment shader implemented by DPC 606 .

[0098] In at least one embodiment, each DPC 606 included in the GPC 600 includes an M-pipe controller ("MPC") 610; a primitive engine 612; one or more SMs 614; and any suitable combination thereof. In at least one embodiment, the MPC 610 controls the operation of the DPC 606, routing packets received from the pipeline manager 602 to appropriate units within the DPC 606. In at least one embodiment, packets associated with vertices are routed to the primitive engine 612, which is configured to retrieve vertex attributes associated with the vertices from memory; conversely, packets associated with shading programs may be sent to the SM 614.

[0099] In at least one embodiment, SM 614 includes a programmable streaming processor configured to process tasks represented by multiple threads. In at least one embodiment, SM 614 is multithreaded and configured to concurrently execute multiple threads (e.g., 32 threads) from a particular thread group and implement a single instruction, multiple data ("SIMD") architecture, wherein each thread in a thread group (e.g., a warp) is configured to process a different data set based on the same instruction set. In at least one embodiment, all threads in a thread group execute a common instruction set. In at least one embodiment, SM 614 implements a single instruction, multiple thread ("SIMT") architecture, wherein each thread in a thread group is configured to process a different data set based on a common instruction set, but wherein individual threads in a thread group are allowed to diverge during execution. In at least one embodiment, a program counter, call stack, and execution state are maintained for each warp, thereby enabling simultaneous execution between warps and serial execution within a warp when threads within the warp diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, enabling concurrency between warps and serial execution within a warp when threads within the warp diverge. In at least one embodiment, execution state is maintained for each individual thread, and threads executing common instructions can be converged and executed in parallel to improve efficiency. At least one embodiment of SM 614 is described in more detail herein.

[0100] In at least one embodiment, MMU 618 provides communication between GPC 600 and memory partitioning unit (e.g., Figure 5 The MMU 618 interfaces between the virtual and physical addresses, memory protection, and memory request arbitration. In at least one embodiment, the MMU 618 provides one or more translation lookaside buffers ("TLBs") for performing translations from virtual addresses to physical addresses in memory.

[0101] The software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate may be provided in Figure 6 A system is used to perform software compilation operations in an embodiment of the present invention, including generating and optimizing peel loops to execute atomic memory update instructions on execution units.

[0102] Figure 7A memory partition unit 700 of a parallel processing unit ("PPU") is shown, according to at least one embodiment. In at least one embodiment, the memory partition unit 700 includes a raster operations ("ROP") unit 702, a level 2 ("L2") cache 704, a memory interface 706, and any suitable combination thereof. In at least one embodiment, the memory interface 706 is coupled to a memory. In at least one embodiment, the memory interface 706 can implement a 32-bit, 64-bit, 17-bit, 1024-bit data bus, etc., for high-speed data transfer. In at least one embodiment, the PPU incorporates U memory interfaces 706, where U is a positive integer, one for each pair of partition units 700, wherein each pair of partition units 700 is connected to a corresponding memory device. For example, in at least one embodiment, the PPU can be connected to up to Y memory devices, such as a high-bandwidth memory stack or Graphics Double Data Rate version 5 synchronous dynamic random access memory ("GDDR5 SDRAM").

[0103] In at least one embodiment, the memory interface 706 implements a high bandwidth memory second generation ("HBM2") memory interface, and Y is equal to half of U. In at least one embodiment, the HBM2 memory stack is located on a physical package with a PPU, providing significant power and area savings compared to traditional GDDR5 SDRAM systems. In at least one embodiment, each HBM2 stack includes four memory tube cores with Y=4, and each tube core of each HBM2 stack includes two 128-bit channels, for a total of 8 channels and a data bus width of 1024 bits. In at least one embodiment, the memory supports single error correction and double error detection ("SECDED") error correction code ("ECC") to protect data. In at least one embodiment, ECC can provide higher reliability for computing applications that are sensitive to data corruption.

[0104] In at least one embodiment, the PPU implements a multi-level memory hierarchy. In at least one embodiment, the memory partitioning unit 700 supports unified memory to provide a single unified virtual address space for the central processing unit ("CPU") and PPU memory, thereby enabling data sharing between virtual memory systems. In at least one embodiment, the frequency of PPU accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU that accesses the pages more frequently. In at least one embodiment, the high-speed GPU interconnect 508 supports address translation services, allowing the PPU to directly access the CPU's page tables and providing full access to the CPU's memory through the PPU.

[0105] In at least one embodiment, the copy engine transfers data between multiple PPUs or between a PPU and a CPU. In at least one embodiment, the copy engine can generate a page fault for an address that is not mapped in the page table, and the memory partition unit 700 then services the page fault, maps the address into the page table, and then the copy engine performs the transfer. In at least one embodiment, memory is pinned (e.g., non-pageable) for multiple copy engine operations between multiple processors, thereby significantly reducing available memory. In at least one embodiment, in the event of a hardware page fault, the address can be passed to the copy engine regardless of whether the memory page is resident, and the copy process is transparent.

[0106] According to at least one embodiment, Figure 5 Data from the memory 504 or other system memory is retrieved by the memory partition unit 700 and stored in the L2 cache 704, which is located on the chip and shared between the various GPCs. In at least one embodiment, each memory partition unit 700 includes at least a portion of the L2 cache associated with the corresponding memory device. In at least one embodiment, lower-level caches are implemented in various units within the GPC. In at least one embodiment, Figure 6 Each SM 614 in the SM 614 may implement a level 1 ("L1") cache, where the L1 cache is private memory dedicated to a particular SM 614, and data from the L2 cache 704 is fetched and stored in each L1 cache for processing in the functional units of the SM 614. In at least one embodiment, the L2 cache 704 is coupled to Figure 5 Memory interface 706 and XBar 520 are shown.

[0107] In at least one embodiment, ROP unit 702 performs graphics raster operations associated with pixel color, such as color compression, pixel blending, and the like. In at least one embodiment, ROP unit 702 implements depth testing in conjunction with raster engine 608, receiving the depth of a sample location associated with a pixel fragment from the culling engine of raster engine 608. In at least one embodiment, the depth is tested against the corresponding depth in the depth buffer for the sample location associated with the fragment. In at least one embodiment, if the fragment passes the depth test for the sample location, ROP unit 702 updates the depth buffer and sends the result of the depth test to raster engine 608. It should be understood that the number of partition units 700 can be different than the number of GPCs, and thus, in at least one embodiment, each ROP unit 702 can be coupled to each GPC. In at least one embodiment, ROP unit 702 tracks packets received from different GPCs and determines whether the results generated by ROP unit 704 should be routed through XBar 520.

[0108] Figure 8 Streaming Multiprocessor ("SM") 800 is shown in accordance with at least one embodiment. In at least one embodiment, SM 800 is Figure 6 In at least one embodiment, SM 800 includes an instruction cache 802, one or more scheduler units 804, registers 808, one or more processing cores ("cores") 810, one or more special function units ("SFUs") 812, one or more load / store units ("LSUs") 814, an interconnection network 816, a shared memory / level 1 ("L1") cache 818, and / or any suitable combination thereof.

[0109] In at least one embodiment, a work distribution unit schedules tasks for execution on a general processing cluster ("GPC") of a parallel processing unit ("PPU"), with each task being assigned to a specific data processing cluster ("DPC") within the GPC. If the task is associated with a shader program, the task is assigned to one of the SMs 800. In at least one embodiment, a scheduler unit 804 receives tasks from the work distribution unit and manages the scheduling of instructions for one or more thread blocks assigned to the SMs 800. In at least one embodiment, the scheduler unit 804 schedules the thread blocks for execution as warps of parallel threads, where each thread block is assigned at least one warp. In at least one embodiment, each warp executes a thread. In at least one embodiment, the scheduler unit 804 manages a plurality of different thread blocks, assigning warps to different thread blocks, and then dispatching instructions from a plurality of different cooperative groups to various functional units (e.g., processing cores 810, SFUs 812, and LSUs 814) during each clock cycle.

[0110] In at least one embodiment, cooperative groups can refer to a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, enabling the expression of richer, more efficient parallel decompositions. In at least one embodiment, a cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. In at least one embodiment, conventional programming models provide a single, simple structure for synchronizing cooperating threads: a barrier across all threads of a thread block (e.g., the syncthreads() function). However, in at least one embodiment, programmers often want to define thread groups at a granularity smaller than the thread block granularity and synchronize within the defined group, enabling higher performance, design flexibility, and software reuse in the form of a collective group-wide function interface. In at least one embodiment, cooperative groups enable programmers to explicitly define thread groups at sub-block (e.g., as small as a single thread) and multi-block granularity and perform collective operations, such as synchronization on threads in the cooperative group. In at least one embodiment, the programming model supports clean composition across software boundaries so that libraries and utility functions can safely synchronize in their local environment without making assumptions about convergence. In at least one embodiment, the cooperative group primitive enables new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across an entire grid of thread blocks.

[0111] In at least one embodiment, the dispatch unit 806 is configured to send instructions to one or more functional units and the scheduler unit 804 and includes two dispatch units 806, which enables scheduling two different instructions from the same warp during each clock cycle. In at least one embodiment, each scheduler unit 804 includes a single dispatch unit 806 or additional dispatch units 806.

[0112] In at least one embodiment, each SM 800 includes registers 808 that provide a set of registers for the functional units of SM 800. In at least one embodiment, registers 808 are partitioned between each functional unit, such that each functional unit is allocated a dedicated portion of registers 808. In at least one embodiment, registers 808 are partitioned between different warps executed by SM 800, and registers 808 provide temporary storage for operands connected to the data paths of the functional units. In at least one embodiment, each SM 800 includes a plurality of L processing cores 810, where L is a positive integer. In at least one embodiment, SM 800 includes a large number (e.g., 128 or more) of different processing cores 810. In at least one embodiment, each processing core 810 includes a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, including a floating-point arithmetic logic unit and an integer arithmetic logic unit. In at least one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point operations. In at least one embodiment, processing core 810 includes 64 single-precision (32-bit) floating point cores, 64 integer cores, 32 double-precision (64-bit) floating point cores, and 8 tensor cores.

[0113] According to at least one embodiment, the tensor core is configured to perform matrix operations. In at least one embodiment, the processing core 810 includes one or more tensor cores. In at least one embodiment, the tensor core is configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In at least one embodiment, each tensor core operates on a 4×4 matrix and performs a matrix multiplication and accumulation operation D=A×B+C, where A, B, C, and D are 4×4 matrices.

[0114] In at least one embodiment, the matrix multiplication inputs A and B are 16-bit floating-point matrices, and the accumulation matrices C and D are 16-bit or 32-bit floating-point matrices. In at least one embodiment, the tensor cores operate on 16-bit floating-point input data and 32-bit floating-point accumulations. In at least one embodiment, the 16-bit floating-point multiplication requires 64 operations, producing a full-precision product, which is then accumulated using 32-bit floating-point additions with other intermediate products of the 4×4×4 matrix multiplication. In at least one embodiment, the tensor cores are used to perform larger two-dimensional or higher-dimensional matrix operations built from these smaller elements. In at least one embodiment, an API (such as the CUDA9 C++ API) exposes specialized matrix load, matrix multiplication and accumulation, and matrix store operations to efficiently use the tensor cores from a CUDA-C++ program. In at least one embodiment, at the CUDA level, the warp-level interface assumes that a 16×16 size matrix spans all 32 threads of the warp.

[0115] In at least one embodiment, each SM 800 includes M SFUs 812 that perform special functions (e.g., attribute evaluation, reciprocal square root, etc.). In at least one embodiment, the SFUs 812 include a tree traversal unit configured to traverse a hierarchical tree data structure. In at least one embodiment, the SFUs 812 include a texture unit configured to perform texture map filtering operations. In at least one embodiment, the texture unit is configured to load a texture map (e.g., a 2D array of texels) from memory and sample the texture map to generate sampled texture values ​​for use in a shader program executed by the SM 800. In at least one embodiment, the texture map is stored in shared memory / L1 cache 818. According to at least one embodiment, the texture unit uses mip-maps (e.g., texture maps of different levels of detail) to implement texture operations such as filtering operations. In at least one embodiment, each SM 800 includes two texture units.

[0116] In at least one embodiment, each SM 800 includes N LSUs 814, which perform load and store operations between shared memory / L1 cache 818 and registers 808. In at least one embodiment, an interconnection network 816 connects each functional unit to registers 808 and connects the LSUs 814 to registers 804 and shared memory / L1 cache 818. In at least one embodiment, the interconnection network 816 is a crossbar switch that can be configured to connect any functional unit to any register in registers 808 and to connect the LSUs 814 to memory locations in registers 808 / L1 cache 818.

[0117] In at least one embodiment, shared memory / L1 cache 818 is an on-chip memory array that, in at least one embodiment, allows for data storage and communication between the SM 800 and the primitive engines, as well as between threads within the SM 800. In at least one embodiment, shared memory / L1 cache 818 includes 128KB of storage capacity and is in the path from the SM 800 to the partition unit. In at least one embodiment, shared memory / L1 cache 818, in at least one embodiment, is used to cache reads and writes. In at least one embodiment, one or more of shared memory / L1 cache 818, L2 cache, and memory is a backing store.

[0118] In at least one embodiment, combining data cache and shared memory functionality into a single memory block provides improved performance for both types of memory accesses. In at least one embodiment, this capacity can be used by programs as a cache that does not utilize shared memory, such as if the shared memory is configured to use half of its capacity, and texture and load / store operations can use the remaining capacity. According to at least one embodiment, the integration within the shared memory / L1 cache 818 enables the shared memory / L1 cache 818 to function as a high-throughput pipeline for streaming data, while providing high-bandwidth and low-latency access to frequently reused data. In at least one embodiment, when configured for general-purpose parallel computing, a simpler configuration can be used compared to graphics processing. In at least one embodiment, the fixed-function graphics processing unit is bypassed, creating a simpler programming model. In at least one embodiment, in a general-purpose parallel computing configuration, the work distribution unit assigns thread blocks directly to the DPC. In at least one embodiment, the threads in a block execute a common program, use unique thread IDs in computations to ensure that each thread generates unique results, use SM 800 to execute the program and perform computations, use shared memory / L1 cache 818 to communicate between threads, and use LSU 814 to read and write global memory through shared memory / L1 cache 818 and a memory partitioning unit. In at least one embodiment, when configured for general-purpose parallel computation, SM 800 writes commands that scheduler unit 804 can use to start new work on a DPC.

[0119] In at least one embodiment, the PPU is included in or coupled to a desktop computer, laptop computer, tablet computer, server, supercomputer, smartphone (e.g., wireless handheld device), personal digital assistant ("PDA"), digital camera, vehicle, head-mounted display, handheld electronic device, etc. In at least one embodiment, the PPU is implemented on a single semiconductor substrate. In at least one embodiment, the PPU is included on a system-on-chip ("SoC") along with one or more other devices (such as an additional PPU, memory, a reduced instruction set computer ("RISC") CPU, a memory management unit ("MMU"), a digital-to-analog converter ("DAC"), etc.

[0120] In at least one embodiment, the PPU can be included on a graphics card that includes one or more memory devices. In at least one embodiment, the graphics card can be configured to interface with a PCIe slot on a motherboard of a desktop computer. In at least one embodiment, the PPU can include an integrated graphics processing unit ("iGPU") in a chipset of the motherboard.

[0121] The software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate may be provided in Figure 8 A system is used to perform software compilation operations in an embodiment of the present invention, including generating and optimizing peel loops to execute atomic memory update instructions on execution units.

[0122] Figure 9 is a block diagram illustrating a computing system 900 according to at least one embodiment. In at least one embodiment, computing system 900 includes a processing subsystem 901 having one or more processors 902 and system memory 904 communicating via an interconnect path that may include a memory hub 905. In at least one embodiment, memory hub 905 may be a separate component within a chipset assembly or may be integrated within one or more processors 902. In at least one embodiment, memory hub 905 is coupled to an I / O subsystem 911 via a communication link 906. In at least one embodiment, I / O subsystem 911 includes an I / O hub 907, which may enable computing system 900 to receive input from one or more input devices 908. In at least one embodiment, I / O hub 907 may enable a display controller, which may be included in one or more processors 902, to provide output to one or more display devices 910A. In at least one embodiment, the one or more display devices 910A coupled to I / O hub 907 may include local, internal, or embedded display devices.

[0123] In at least one embodiment, the processing subsystem 901 includes one or more parallel processors 912 coupled to a memory hub 905 via a bus or other communication link 913. In at least one embodiment, the communication link 913 can use one of any number of standard based communication link technologies or protocols (such as, but not limited to, PCI Express), or can be a vendor-specific communication interface or communication structure. In at least one embodiment, the one or more parallel processors 912 form a parallel or vector processing system in a computational focus, which can include a large number of processing cores and / or processing clusters, such as an integrated many-core (MIC) processor. In at least one embodiment, some or all of the one or more parallel processors 912 form a graphics processing subsystem that can output pixels to one of one or more display devices 910A coupled via an I / O hub 907. In at least one embodiment, the one or more parallel processors 912 can also include a display controller and display interface (not shown) for enabling direct connection to one or more display devices 910B.

[0124] In at least one embodiment, a system storage unit 914 can be connected to the I / O hub 907 to provide a storage mechanism for the computing system 900. In at least one embodiment, an I / O switch 916 can be used to provide an interface mechanism for enabling connections between the I / O hub 907 and other components, such as a network adapter 918 and / or a wireless network adapter 919 that can be integrated into the platform, as well as various other devices that can be added via one or more add-on devices 920. In at least one embodiment, the network adapter 918 can be an Ethernet adapter or another wired network adapter. In at least one embodiment, the wireless network adapter 919 can include one or more of Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more radio devices.

[0125] In at least one embodiment, the computing system 900 may include other components not explicitly shown that may also be connected to the I / O hub 907, including USB or other port connections, optical storage drives, video capture devices, etc. In at least one embodiment, the interconnection may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI-Express) or other bus or point-to-point communication interface and / or protocol (such as the NV-Link high-speed interconnect or interconnect protocol). Figure 9 The communication paths between the various components in the system.

[0126] In at least one embodiment, one or more parallel processors 912 include circuits optimized for graphics and video processing, including, for example, video output circuitry, and constitute a graphics processing unit (GPU). In at least one embodiment, one or more parallel processors 912 include circuits optimized for general-purpose processing. In at least one embodiment, the components of the computing system 900 can be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, one or more parallel processors 912, the memory hub 905, one or more processors 902, and the I / O hub 907 can be integrated into a system-on-chip (SoC) integrated circuit. In at least one embodiment, the components of the computing system 900 can be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least a portion of the components of the computing system 900 can be integrated into a multi-chip module (MCM), which can be interconnected with other multi-chip modules into a modular computing system.

[0127] Software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate can be used to Figure 9 A system for performing software compilation operations, including generating and optimizing a peel loop to execute atomic memory update instructions on an execution unit.

[0128] Figure 10A 1 shows a parallel processor 1000 according to at least one embodiment. In at least one embodiment, the various components of the parallel processor 1000 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). In at least one embodiment, the parallel processor 1000 shown is Figure 9 A variation of the one or more parallel processors 912 is shown.

[0129] In at least one embodiment, parallel processor 1000 includes parallel processing unit 1002. In at least one embodiment, parallel processing unit 1002 includes an I / O unit 1004 that enables communication with other devices, including other instances of parallel processing unit 1002. In at least one embodiment, I / O unit 1004 can be directly connected to other devices. In at least one embodiment, I / O unit 1004 connects to other devices using a hub or switch interface (e.g., memory hub 1005). In at least one embodiment, the connection between memory hub 1005 and I / O unit 1004 forms a communication link 1013. In at least one embodiment, I / O unit 1004 is connected to a host interface 1006 and a memory crossbar switch 1016, where host interface 1006 receives commands for performing processing operations and memory crossbar switch 1016 receives commands for performing memory operations.

[0130] In at least one embodiment, when host interface 1006 receives command buffers via I / O unit 1004, host interface 1006 can direct work operations for executing those commands to front end 1008. In at least one embodiment, front end 1008 is coupled to scheduler 1010, which is configured to distribute commands or other work items to processing cluster array 1012. In at least one embodiment, scheduler 1010 ensures that processing cluster array 1012 is properly configured and in a valid state before distributing tasks to clusters within processing cluster array 1012. In at least one embodiment, scheduler 1010 is implemented via firmware logic executing on a microcontroller. In at least one embodiment, a microcontroller-implemented scheduler 1010 can be configured to perform complex scheduling and work distribution operations at both coarse and fine granularity, thereby enabling fast preemption and context switching of threads executing on processing cluster array 1012. In at least one embodiment, host software can identify workloads for scheduling on processing cluster array 1012 via one of multiple graphics processing paths. In at least one embodiment, the workload may then be automatically distributed across the processing cluster array 1012 by scheduler 1010 logic within a microcontroller that includes scheduler 1010 .

[0131] In at least one embodiment, processing cluster array 1012 may include up to "N" processing clusters (e.g., cluster 1014A, cluster 1014B, through cluster 1014N), where "N" represents a positive integer (which may be an integer "N" different from the integers used in other figures). In at least one embodiment, each cluster 1014A-1014N in processing cluster array 1012 may execute a large number of concurrent threads. In at least one embodiment, scheduler 1010 may use various scheduling and / or work distribution algorithms to distribute work to clusters 1014A-1014N in processing cluster array 1012, which may vary depending on the workload generated for each type of program or computation. In at least one embodiment, scheduling may be handled dynamically by scheduler 1010 or may be partially assisted by compiler logic during the compilation of program logic configured to be executed by processing cluster array 1012. In at least one embodiment, different clusters 1014A-1014N in processing cluster array 1012 may be assigned to process different types of programs or to perform different types of computations.

[0132] In at least one embodiment, processing cluster array 1012 can be configured to perform various types of parallel processing operations. In at least one embodiment, processing cluster array 1012 is configured to perform general-purpose parallel computing operations. For example, in at least one embodiment, processing cluster array 1012 can include logic for performing processing tasks including filtering video and / or audio data, performing modeling operations including physics operations, and performing data transformations.

[0133] In at least one embodiment, processing cluster array 1012 is configured to perform parallel graphics processing operations. In at least one embodiment, processing cluster array 1012 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic to perform texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, processing cluster array 1012 may be configured to execute shader programs related to graphics processing, such as vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. In at least one embodiment, parallel processing units 1002 may transfer data from system memory via I / O units 1004 for processing. In at least one embodiment, during processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 1022) during processing and then written back to system memory.

[0134] In at least one embodiment, when parallel processing unit 1002 is used to perform graphics processing, scheduler 1010 can be configured to divide the processing workload into tasks of approximately equal size to better facilitate the distribution of graphics processing operations to multiple clusters 1014A-1014N in processing cluster array 1012. In at least one embodiment, different portions of processing cluster array 1012 can be configured to perform different types of processing. For example, in at least one embodiment, a first portion can be configured to perform vertex shading and topology generation, a second portion can be configured to perform tessellation and geometry shading, and a third portion can be configured to perform pixel shading or other screen-space operations to produce a rendered image for display. In at least one embodiment, intermediate data generated by one or more of clusters 1014A-1014N can be stored in a buffer to allow the intermediate data to be transferred between clusters 1014A-1014N for further processing.

[0135] In at least one embodiment, the processing cluster array 1012 can receive processing tasks to be executed via the scheduler 1010, which receives commands defining the processing tasks from the front end 1008. In at least one embodiment, the processing tasks can include an index of data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands defining how to process the data (e.g., what program to execute). In at least one embodiment, the scheduler 1010 can be configured to obtain the index corresponding to the task, or can receive the index from the front end 1008. In at least one embodiment, the front end 1008 can be configured to ensure that the processing cluster array 1012 is configured in a valid state before starting the workload specified by the incoming command buffer (e.g., batch buffer, push buffer, etc.).

[0136] In at least one embodiment, each of one or more instances of parallel processing unit 1002 can be coupled to parallel processor memory 1022. In at least one embodiment, parallel processor memory 1022 can be accessed via memory crossbar 1016, which can receive memory requests from processing cluster array 1012 and I / O unit 1004. In at least one embodiment, memory crossbar 2516 can access parallel processor memory 1022 via memory interface 1018. In at least one embodiment, memory interface 1018 can include a plurality of partition units (e.g., partition unit 1020A, partition unit 1020B, through partition unit 1020N), each of which can be coupled to a portion (e.g., a memory unit) of parallel processor memory 1022. In at least one embodiment, the number of partition units 1020A-1020N is configured to be equal to the number of memory cells, such that the first partition unit 1020A has a corresponding first memory cell 1024A, the second partition unit 1020B has a corresponding second memory cell 1024B, and the Nth partition unit 1020N has a corresponding Nth memory cell 1024N. In at least one embodiment, the number of partition units 1020A-1020N may not be equal to the number of memory cells.

[0137] In at least one embodiment, memory units 1024A-1024N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory units 1024A-1024N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). In at least one embodiment, render targets such as frame buffers or texture maps may be stored across memory units 1024A-1024N, allowing partition units 1020A-1020N to write to portions of each render target in parallel to efficiently use the available bandwidth of parallel processor memory 1022. In at least one embodiment, local instances of parallel processor memory 1022 may be eliminated in favor of a unified memory design that utilizes system memory as well as local cache memory.

[0138] In at least one embodiment, any of the clusters 1014A-1014N in the processing cluster array 1012 can process data to be written to any memory unit 1024A-1024N within the parallel processor memory 1022. In at least one embodiment, the memory crossbar 1016 can be configured to transmit the output of each cluster 1014A-1014N to any partition unit 1020A-1020N or to another cluster 1014A-1014N, which can perform additional processing operations on the output. In at least one embodiment, each cluster 1014A-1014N can communicate with a memory interface 1018 via the memory crossbar 1016 to read from or write to various external memory devices. In at least one embodiment, memory crossbar 1016 has connections to memory interface 1018 for communicating with I / O unit 1004, and connections to a local instance of parallel processor memory 1022, which enables processing units within different processing clusters 1014A-1014N to communicate with system memory or other memory that is not local to parallel processing unit 1002. In at least one embodiment, memory crossbar 1016 may use virtual channels to separate traffic flows between clusters 1014A-1014N and partition units 1020A-1020N.

[0139] In at least one embodiment, multiple instances of parallel processing unit 1002 can be provided on a single add-in card, or multiple add-in cards can be interconnected. In at least one embodiment, different instances of parallel processing unit 1002 can be configured to interoperate with each other, even if the different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. For example, in at least one embodiment, some instances of parallel processing unit 1002 can include a higher precision floating point unit than other instances. In at least one embodiment, a system including one or more instances of parallel processing unit 1002 or parallel processor 1000 can be implemented in a variety of configurations and form factors, including but not limited to desktop computers, laptop computers, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0140] Figure 10B is a block diagram of a partition unit 1020 according to at least one embodiment. In at least one embodiment, the partition unit 1020 is Figure 10A1020N。In at least one embodiment, the partition unit 1020 includes an L2 cache 1021, a frame buffer interface 1025 and an ROP 1026 (raster operation unit). In at least one embodiment, the L2 cache 1021 is a read / write cache that is configured to perform load and store operations received from the memory crossbar switch 1016 and the ROP 1026. In at least one embodiment, the L2 cache 1021 outputs read misses and urgent write-back requests to the frame buffer interface 1025 for processing. In at least one embodiment, updates can also be sent to the frame buffer via the frame buffer interface 1025 for processing. In at least one embodiment, the frame buffer interface 1025 communicates with memory units in the parallel processor memory (such as Figure 10A One of the memory units 1024A-1024N (e.g., within parallel processor memory 1022)) is coupled.

[0141] In at least one embodiment, ROP 1026 is a processing unit that performs raster operations such as stenciling, z-testing, blending, etc. In at least one embodiment, ROP 1026 then outputs the processed graphics data that is stored in graphics memory. In at least one embodiment, ROP 1026 includes compression logic for compressing depth or color data written to memory and decompressing depth or color data read from memory. In at least one embodiment, the compression logic can be lossless compression logic that utilizes one or more of a variety of compression algorithms. In at least one embodiment, the type of compression performed by ROP 1026 can vary based on the statistical characteristics of the data to be compressed. For example, in at least one embodiment, incremental color compression is performed on depth and color data on a per-tile basis.

[0142] In at least one embodiment, ROP 1026 is included within each processing cluster (e.g., Figure 10A In at least one embodiment, read and write requests for pixel data, rather than pixel fragment data, are routed through memory crossbar 1016. In at least one embodiment, the processed graphics data may be displayed on a display device such as a Figure 9 910), routed by processor 902 for further processing, or by Figure 10A One of the processing entities within parallel processor 1000 is routed for further processing.

[0143] Figure 10C is a block diagram of a processing cluster 1014 within a parallel processing unit according to at least one embodiment. In at least one embodiment, a processing cluster is Figure 10AIn at least one embodiment, processing cluster 1014 can be configured to execute many threads in parallel, where a "thread" refers to an instance of a particular program executed on a particular set of input data. In at least one embodiment, single instruction multiple data (SIMD) instruction issuance technology is used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, single instruction multiple thread (SIMT) technology is used to support the parallel execution of a large number of generally synchronized threads using a common instruction unit that is configured to issue instructions to a set of processing engines within each processing cluster.

[0144] In at least one embodiment, the operation of the processing cluster 1014 can be controlled via a pipeline manager 1032 that distributes processing tasks to SIMT parallel processors. In at least one embodiment, the pipeline manager 1032 Figure 10A The scheduler 1010 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1034 and / or the texture unit 1036. In at least one embodiment, the graphics multiprocessor 1034 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, various types of SIMT parallel processors with different architectures may be included within the processing cluster 1014. In at least one embodiment, one or more instances of the graphics multiprocessor 1034 may be included within the processing cluster 1014. In at least one embodiment, the graphics multiprocessor 1034 may process data, and the data crossbar 1040 may be used to distribute the processed data to one of multiple possible destinations (including other shader units). In at least one embodiment, the pipeline manager 1032 may facilitate the distribution of the processed data by specifying the destination of the processed data to be distributed via the data crossbar 1040.

[0145] In at least one embodiment, each graphics multiprocessor 1034 within a processing cluster 1014 may include the same set of function execution logic (e.g., arithmetic logic unit, load-store unit, etc.). In at least one embodiment, the function execution logic may be configured in a pipelined manner, where new instructions may be issued before previous instructions have completed. In at least one embodiment, the function execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifts, and calculations of various algebraic functions. In at least one embodiment, the same functional unit hardware may be utilized to perform different operations, and any combination of functional units may be present.

[0146] In at least one embodiment, instructions transmitted to processing cluster 1014 constitute threads. In at least one embodiment, a group of threads executed across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a common program on different input data. In at least one embodiment, each thread within a thread group can be assigned to a different processing engine within graphics multiprocessor 1034. In at least one embodiment, a thread group can include fewer threads than the number of processing engines within graphics multiprocessor 1034. In at least one embodiment, when a thread group includes fewer threads than the number of processing engines, one or more processing engines can be idle during the cycle in which the thread group is being processed. In at least one embodiment, a thread group can also include more threads than the number of processing engines within graphics multiprocessor 1034. In at least one embodiment, when a thread group includes more threads than the number of processing engines within graphics multiprocessor 1034, processing can be performed in consecutive clock cycles. In at least one embodiment, multiple thread groups can be executed concurrently on graphics multiprocessor 1034.

[0147] In at least one embodiment, graphics multiprocessor 1034 includes internal cache memory for performing load and store operations. In at least one embodiment, graphics multiprocessor 1034 can abandon the internal cache and use cache memory within processing cluster 1014 (e.g., L1 cache 1048). In at least one embodiment, each graphics multiprocessor 1034 can also access partition units (e.g., Figure 10A 1020N) are shared across all processing clusters 1014 and can be used to transfer data between threads. In at least one embodiment, graphics multiprocessor 1034 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory external to parallel processing unit 1002 can be used as global memory. In at least one embodiment, processing cluster 1014 includes multiple instances of graphics multiprocessor 1034, which can share common instructions and data, which can be stored in L1 cache 1048.

[0148] In at least one embodiment, each processing cluster 1014 may include a memory management unit ("MMU") 1045 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of MMU 1045 may reside in Figure 10A1048 . In at least one embodiment, the MMU 1045 includes a set of page table entries (PTEs) that are used to map virtual addresses to physical addresses of tiles and optionally to cache line indices. In at least one embodiment, the MMU 1045 may include an address translation lookaside buffer (TLB) or a cache that may reside within the graphics multiprocessor 1034 or the L1 cache 1048 or the processing cluster 1014. In at least one embodiment, the physical addresses are processed to assign surface data access locality for efficient request interleaving between partition units. In at least one embodiment, the cache line index may be used to determine whether a request for a cache line is a hit or a miss.

[0149] In at least one embodiment, processing clusters 1014 can be configured such that each graphics multiprocessor 1034 is coupled to a texture unit 1036 to perform texture mapping operations, which determine texture sample locations, read texture data, and filter texture data. In at least one embodiment, texture data is read from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 1034, and retrieved from an L2 cache, local parallel processor memory, or system memory, as needed. In at least one embodiment, each graphics multiprocessor 1034 outputs processed tasks to a data crossbar 1040 to provide the processed tasks to another processing cluster 1014 for further processing, or to store the processed tasks in an L2 cache, local parallel processor memory, or in system memory via a memory crossbar 1016. In at least one embodiment, a preROP 1042 (pre-raster operations unit) is configured to receive data from the graphics multiprocessor 1034 and direct the data to a ROP unit, which can communicate with a partition unit (e.g., a partition unit) as described herein. Figure 10A In at least one embodiment, the PreROP 1042 unit can perform optimizations for color blending, organizing pixel color data, and performing address translation.

[0150] Software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate can be used to Figure 10C A system for performing software compilation operations, including generating and optimizing a peel loop to execute atomic memory update instructions on an execution unit.

[0151] Figure 10D A graphics multiprocessor 1034 is shown in accordance with at least one embodiment. In at least one embodiment, the graphics multiprocessor 1034 is coupled to a pipeline manager 1032 of the processing cluster 1014. In at least one embodiment, the graphics multiprocessor 1034 has an execution pipeline that includes, but is not limited to, an instruction cache 1052, an instruction unit 1054, an address mapping unit 1056, registers 1058, one or more general purpose graphics processing unit (GPGPU) cores 1062, and one or more load / store units 1066. In at least one embodiment, the GPGPU cores 1062 and the load / store units 1066 are coupled to a cache memory 1072 and a shared memory 1070 via a memory and cache interconnect 1068.

[0152] In at least one embodiment, the instruction cache 1052 receives a stream of instructions to be executed from the pipeline manager 1032. In at least one embodiment, the instructions are cached in the instruction cache 1052 and dispatched for execution by the instruction unit 1054. In at least one embodiment, the instruction unit 1054 can dispatch instructions as thread groups (e.g., warps), where each thread in the thread group is assigned to a different execution unit within the GPGPU core 1062. In at least one embodiment, the instructions can access any local, shared, or global address space by specifying an address within the unified address space. In at least one embodiment, the address mapping unit 1056 can be used to convert addresses in the unified address space into different memory addresses that can be accessed by the load / store unit 1066.

[0153] In at least one embodiment, registers 1058 provide a set of registers for the functional units of graphics multiprocessor 1034. In at least one embodiment, registers 1058 provide temporary storage for operands for data paths connected to the functional units of graphics multiprocessor 1034 (e.g., GPGPU core 1062, load / store unit 1066). In at least one embodiment, registers 1058 are divided between each functional unit such that a dedicated portion of registers 1058 is allocated to each functional unit. In at least one embodiment, registers 1058 are divided between different warps being executed by graphics multiprocessor 1034.

[0154] In at least one embodiment, each GPGPU core 1062 may include a floating-point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1034. In at least one embodiment, the architectures of the various GPGPU cores 1062 may be similar or different. In at least one embodiment, a first portion of a GPGPU core 1062 includes a single-precision FPU and integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. In at least one embodiment, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic. In at least one embodiment, the graphics multiprocessor 1034 may additionally include one or more fixed-function or special-function units for performing specific functions, such as copying rectangles or pixel blending operations. In at least one embodiment, one or more of the GPGPU cores 1062 may also include fixed-function or special-function logic.

[0155] In at least one embodiment, the GPGPU core 1062 includes SIMD logic capable of executing a single instruction on multiple sets of data. In at least one embodiment, the GPGPU core 1062 can physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. In at least one embodiment, the SIMD instructions for the GPGPU core can be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. In at least one embodiment, multiple threads of a program configured for a SIMT execution model can be executed via a single SIMD instruction. For example, in at least one embodiment, eight SIMT threads that implement the same or similar operations can be executed in parallel via a single SIMD8 logic unit.

[0156] In at least one embodiment, memory and cache interconnect 1068 is an interconnect network that connects each functional unit of graphics multiprocessor 1034 to registers 1058 and shared memory 1070. In at least one embodiment, memory and cache interconnect 1068 is a crossbar interconnect that allows load / store unit 1066 to perform load and store operations between shared memory 1070 and registers 1058. In at least one embodiment, registers 1058 can operate at the same frequency as GPGPU core 1062, resulting in very low latency for data transfers between GPGPU core 1062 and registers 1058. In at least one embodiment, shared memory 1070 can be used to facilitate communication between threads executing on functional units within graphics multiprocessor 1034. In at least one embodiment, cache memory 1072 can be used, for example, as a data cache to cache texture data communicated between functional units and texture unit 1036. In at least one embodiment, shared memory 1070 can also be used as a program-managed cache. In at least one embodiment, in addition to automatically cached data stored in cache memory 1072, threads executing on GPGPU core 1062 may programmatically store data in shared memory.

[0157] In at least one embodiment, a parallel processor or GPGPU as described herein is communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. In at least one embodiment, the GPU can be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., a high-speed interconnect such as PCIe or NVLink). In at least one embodiment, the GPU can be integrated with the core on a package or chip and communicatively coupled to the core via an internal processor bus / interconnect within the package or chip. In at least one embodiment, regardless of how the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. In at least one embodiment, the GPU then uses dedicated circuitry / logic to efficiently process these commands / instructions.

[0158] Software compiler logic 150 may be used to perform software compilation operations associated with one or more embodiments, including generating and optimizing peel loops to execute atomic memory update instructions on execution units. Figure 1 Details regarding the software compiler logic 150 are provided. In at least one embodiment, the software compiler logic 150 and the program code it can generate can be used to Figure 10DA system for performing software compilation operations, including generating and optimizing a peel loop to execute atomic memory update instructions on an execution unit.

[0159] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. However, it should be understood that there is no intention to limit the disclosure to one or more specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the present disclosure as defined by the appended claims.

[0160] Unless otherwise noted or clearly contradicted by the context, the use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (particularly in the context of the appended claims) should be interpreted as covering the singular and plural, rather than as definitions of terms. Unless otherwise noted, the terms "include," "have," "include," and "contain" should be interpreted as open terms (meaning "including but not limited to"), unless otherwise noted. The term "connected" (which refers to a physical connection when unmodified) should be interpreted as partially or completely contained within, attached to, or connected together, even if there are some intervening objects. Unless otherwise noted herein, references to numerical ranges herein are intended only to be used as a shorthand method of referring to each individual value falling within the range, and each individual value is incorporated into the specification as if it were separately described herein. In at least one embodiment, unless otherwise noted or contradicted by the context, the use of the term "set" (e.g., "item set") or "subset" should be interpreted as a non-empty set including one or more members. Furthermore, unless otherwise indicated or contradicted by context, the term "subset" of a corresponding set does not necessarily mean a proper subset of the corresponding set, but rather a subset and a corresponding set may be equivalent.

[0161] Unless otherwise expressly indicated or clearly contradicted by context, conjunctions such as phrases of the form "at least one of A, B, and C" or "at least one of A, B, and C" are understood in the context to generally refer to an item, term, or the like, which may be A or B or C, or any non-empty subset of the set of A, B, and C. For example, in the illustrative example of a set having three members, the conjunctions "at least one of A, B, and C" and "at least one of A, B, and C" refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctions are not generally intended to imply that certain embodiments require the presence of each of at least one of A, at least one of B, and at least one of C. Additionally, unless otherwise expressly indicated or contradicted by context, the term "plurality" indicates plurality (e.g., "a plurality of items" indicates a plurality of items). In at least one embodiment, the number of items in the plurality of items is at least two, but may be more if expressly indicated or indicated by context. Further, unless stated otherwise or clear from context, the phrase "based on" means "based at least in part on" rather than "based solely on."

[0162] Unless otherwise indicated herein or clearly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as those described herein (or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that are collectively executed by hardware or a combination thereof on one or more processors. In at least one embodiment, the code is stored on a computer-readable storage medium in the form of, for example, a computer program that includes a plurality of instructions that can be executed by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., propagated transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuits (e.g., buffers, caches, and queues) within a transceiver for transient signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) having executable instructions stored thereon, which, when executed by one or more processors of a computer system (e.g., as a result of being executed), causes the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media includes a plurality of non-transitory computer-readable storage media, and one or more of the plurality of non-transitory computer-readable storage media lacks all of the code, but rather the plurality of non-transitory computer-readable storage media collectively store all of the code. In at least one embodiment, the executable instructions are executed so that different instructions are executed by different processors, for example, a non-transitory computer-readable storage medium stores instructions, and a main central processing unit ("CPU") executes some instructions, while a graphics processing unit ("GPU") executes other instructions. In at least one embodiment, different components of the computer system have separate processors, and different processors execute different subsets of instructions.

[0163] Thus, in at least one embodiment, a computer system is configured to implement one or more services that individually or collectively perform the operations of the processes described herein, and such a computer system is configured with applicable hardware and / or software that enables the operations to be performed. Furthermore, the computer system that implements at least one embodiment of the present disclosure is a single device, and in another embodiment is a distributed computer system that includes multiple devices that operate differently, such that the distributed computer system performs the operations described herein, and such that no single device performs all of the operations.

[0164] The use of any and all examples or exemplary language (e.g., "such as") provided herein is intended merely to better illuminate embodiments of the present disclosure and does not impose a limitation on the scope of the disclosure unless otherwise required. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0165] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0166] In the description and claims, the terms "coupled" and "connected," as well as their derivatives, may be used. It should be understood that these terms may not be intended as synonyms for each other. Rather, in specific examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0167] Unless otherwise expressly stated, it is understood that throughout the specification, terms such as “process,” “calculate,” “calculate,” “determine,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulates and / or transforms data represented as physical quantities (e.g., electronic quantities) in the registers and / or memories of the computing system into other data similarly represented as physical quantities in the memories, registers, or other such information storage, transmission, or display devices of the computing system.

[0168] In a similar manner, the term "processor" may refer to any device or portion of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As non-limiting examples, a "processor" may be a CPU or a GPU. A "computing platform" may include one or more processors. As used herein, a "software" process may include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Likewise, each process may refer to multiple processes to execute instructions sequentially or in parallel, continuously or intermittently. In at least one embodiment, the terms "system" and "method" may be used interchangeably herein, as long as a system may embody one or more methods, and a method may be considered a system.

[0169] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data may be accomplished in various ways, such as by receiving data as a parameter of a function call or a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data may be accomplished by transmitting data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data may be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data may be accomplished by transmitting the data as an input or output parameter of a function call, a parameter of an application programming interface, or an inter-process communication mechanism.

[0170] Although the description herein sets forth example implementations of the described technology, other architectures may be used to implement the described functionality and are intended to fall within the scope of this disclosure. In addition, although specific responsibilities are defined above for descriptive purposes, the various functions and responsibilities may be allocated and divided in different ways depending on the circumstances.

[0171] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. A system comprising: Shared memory; as well as One or more processing units coupled to the shared memory, wherein the one or more processing units are configured to: identifying a first set of threads from the active thread group associated with the instruction for updating the shared memory having a same address corresponding to the shared memory; Selecting a first thread from the first set of threads; executing the instruction for the first thread using the same address used to access the shared memory; as well as Attempting to execute the instruction for remaining threads in the first set of threads is delayed until after execution of the first thread is complete and until failure of at least one of the remaining threads in the first set of threads to execute the instruction is not guaranteed.

2. The system of claim 1 , wherein to delay attempting to execute the instruction on remaining threads in the first set of threads, the one or more processing units are to: A Boolean value is stored in one or more predicate registers corresponding to the remaining threads, wherein the Boolean value indicates that the remaining threads have failed to execute the instruction.

3. The system of claim 1 , wherein the one or more processing units are further configured to: In response to execution of the instruction for the first thread, a Boolean value is stored in a predicate register corresponding to the first thread, wherein the Boolean value indicates whether the first thread successfully executed the instruction.

4. The system of claim 1 , wherein the instruction is a compare and store CAST instruction, and wherein to execute the compare and store instruction, the one or more processing units are to: comparing a first value stored at the same memory address of the shared memory with an expected value; and In response to determining that the first value matches the expected value, a second value is written to the shared memory at the same memory address.

5. The system of claim 4, wherein the one or more processing units are further configured to: The second value is written to one or more private registers corresponding to one or more threads of the first set.

6. The system of claim 5, wherein the one or more processing units are further configured to: After execution of the instruction for the first thread, the CAST instruction is executed for a second thread of the remaining threads in the first set of threads using the second value stored in the respective one of the private registers.

7. The system of claim 1 , wherein the shared memory comprises a plurality of logic units, and wherein the one or more processing units are further configured to: The instructions are executed serially for threads from the active thread group having different addresses corresponding to a same logical unit of the plurality of logical units of the shared memory.

8. The system of claim 1 , wherein the system is included in at least one of: control systems for autonomous or semi-autonomous machines; Perception systems for autonomous or semi-autonomous machines; a system for performing simulation operations; Systems for performing digital twin operations; a system for performing light transport simulations; A system for performing collaborative content creation of 3D assets; Systems for performing deep learning operations; a system for generating or presenting at least one of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more live streaming applications; Systems implemented using edge devices; Systems implemented using robots; Systems for performing conversational AI operations; Systems for generating synthetic data; A system comprising one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

9. A method comprising: identifying a first set of threads from the active thread group associated with the instruction for updating the shared memory having a same address corresponding to the shared memory; Selecting a first thread from the first set of threads; executing the instruction for the first thread using the same address used to access the shared memory; as well as Attempting to execute the instruction for remaining threads in the first set of threads is delayed until after execution of the first thread is complete and until failure of at least one of the remaining threads in the first set of threads to execute the instruction is not guaranteed.

10. The method of claim 9, wherein delaying attempting to execute the instruction on the remaining threads in the first set of threads comprises: A Boolean value is stored in one or more predicate registers corresponding to the remaining threads, wherein the Boolean value indicates that the remaining threads have failed to execute the instruction.

11. The method of claim 9, further comprising: In response to execution of the instruction for the first thread, a Boolean value is stored in a predicate register corresponding to the first thread, wherein the Boolean value indicates whether the first thread successfully executed the instruction.

12. The method of claim 9, wherein the instruction is a compare and store CAST instruction, and wherein executing the compare and store instruction comprises: comparing a first value stored at a same memory address in the shared memory with an expected value; as well as In response to determining that the first value matches the expected value, a second value is written to the shared memory at the same memory address.

13. The method of claim 12, further comprising: The second value is written to one or more private registers corresponding to one or more threads of the first set.

14. The method of claim 13, further comprising: After executing the instruction for the first thread, executing the CAST instruction for a second thread of the remaining threads in the first set of threads using the second value stored in the respective one of the private registers.

15. The method of claim 9, wherein the shared memory comprises a plurality of logical units, and wherein the method further comprises: The instructions are executed serially for threads from the active thread group having different addresses corresponding to a same logical unit of the plurality of logical units of the shared memory.

16. A parallel processing unit (PPU), comprising one or more execution units and a shared memory, wherein the PPU is configured to: identifying a first set of threads from the active thread group associated with the instruction for updating the shared memory having a same address corresponding to the shared memory; Selecting a first thread from the first set of threads; executing the instruction for the first thread using the same address used to access the shared memory; as well as Attempting to execute the instruction for the remaining threads in the first set of threads is delayed until after the first thread has completed execution and until at least one of the remaining threads in the first set of threads is not guaranteed to fail execution of the instruction.

17. The PPU of claim 16, wherein to delay attempting to execute the instruction for remaining threads in the first set of threads, the PPU is to: A Boolean value is stored in one or more predicate registers corresponding to the remaining threads, wherein the Boolean value indicates that the remaining threads have failed to execute the instruction.

18. The PPU of claim 16, wherein the PPU is further configured to: In response to execution of the instruction for the first thread, a Boolean value is stored in a predicate register corresponding to the first thread, wherein the Boolean value indicates whether the first thread successfully executed the instruction.

19. The PPU of claim 16, wherein the instruction is a compare-and-store CAST instruction, and wherein to execute the compare-and-store instruction, the PPU is to: comparing a first value stored at the same memory address of the shared memory with an expected value; and In response to determining that the first value matches the expected value, a second value is written to the shared memory at the same memory address.

20. The PPU of claim 19, wherein the PPU is further configured to: The second value is written to one or more private registers corresponding to one or more threads of the first set.

Images