A high-performance synchronization mechanism for coordinating operations on a computer system

Abstract

A high-performance synchronization mechanism for coordinating operations on a computer system is disclosed. A novel synchronization barrier is disclosed for synchronizing operations on a computing system. In one embodiment, the disclosed synchronization barrier provides certain synchronization mechanisms, such as, for example, "arrive" and "wait," that are split to allow for greater flexibility and efficiency in coordinating synchronization. In another embodiment, the disclosed synchronization barrier allows hardware components, such as, for example, dedicated copy or direct memory access (DMA) engines, to synchronize with software-based threads.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Provisional Application No. 62 / 927,417, filed October 29, 2019, and Provisional Application No. 62 / 927,511, filed October 29, 2019, each of which is incorporated herein by reference. This application is related to commonly assigned co-pending U.S. Patent Application No. 16 / 712,083, filed December 12, 2019, which is incorporated herein by reference.

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0004] none Background Art

[0005] Massively parallel, high-performance, multi-threaded, multi-core processing systems—systems containing many processing cores operating in parallel—process data faster than ever before. These processing systems break down complex calculations into smaller tasks that are executed simultaneously by the parallel processing cores. This "divide and conquer" approach can execute complex calculations in a fraction of the time required to perform the same calculations sequentially using only one or a few processors. However, this parallel processing also creates the need for communication and coordination between the parallel executing threads, or blocks.

[0006] One way to coordinate the states of different executing processes is to use barrier synchronization. Barrier synchronization typically involves each process in a set of parallel processes waiting at a barrier until all other processes in the set have caught up. Unless all processes have reached the barrier, no process can advance beyond it.

[0007] Figure 1A-Figure 1H Alongside is a companion flipchart animation illustrating an example of this barrier synchronization. Figure 1A Threads or other execution processes T1, T2, ..., TK are shown. Three threads are shown, but there may be hundreds or thousands of threads. The threads start from a common point (e.g., an initialization point, a previous barrier, etc. - designated here as a "starting line" for illustrative purposes).

[0008] One thread TK is the "hare" which executes faster than the other threads and progresses faster towards the barrier (represented here by a railroad crossing and a graphic "arrive-wait point", the meaning of which will be explained below).

[0009] The other thread, T1, is a "tortoise" - it executes more slowly than the other threads and progresses more slowly towards the barrier.

[0010] Figure 1B 、 1C, 1D shows various threads advancing towards the barrier at different speeds. Figure 1D The "hare" thread TK is shown reaching the barrier before the "tortoise" thread T1. Since the "hare" thread TK cannot cross the barrier until the "tortoise" thread T1 also reaches the barrier, the "hare" thread will wait at the barrier (in the prior art - but see the "split" reach-wait function below). This may involve many delay cycles - for example, if the "tortoise" thread T1 is slow because it is waiting for service from main memory, then the "hare" thread TK may have to wait for a long time before the "tortoise" thread finally catches up and reaches the barrier ( Figure 1E ).

[0011] Once the last loose "turtle" thread T1 reaches the barrier ( Figure 1F ), the barrier will be unblocked (in Figure 1G ), and all threads can cross the barrier.

[0012] An example of a useful application that benefits from synchronization barriers is "asynchronous compute." With asynchronous compute, GPU utilization is improved by scheduling tasks out of order rather than in strict order, so that "later" (in-order) computations can be executed concurrently with "earlier" (in-order) computations. As an example, when rendering graphics, asynchronous compute allows shading execution to be performed concurrently with other work, rather than running shaders sequentially with other workloads. Although GPU APIs can be designed to assume that most or all calls are independent, developers can also control how tasks are scheduled and implement barriers to ensure correctness, such as when one operation depends on the result of another. See, for example, U.S. Patents Nos. 9,117,284 and 10,217,183.

[0013] Hardware-based synchronization mechanisms have been included in GPUs to support such synchronization barrier functionality. See, for example, Xiao et al., "Inter-Block GPU Communication via FastBarrier Synchronization," 2010 IEEE International Symposium on Parallel and Distributed Processing (IPDPS) (April 19-23, 2010). Compute-capable GPUs with such hardware-based synchronization functionality are typically programmed in a batch synchronization fashion—a wide range of parallel tasks with barrier synchronization internally and deriving / joining between them. See, for example, U.S. Patent Publication No. 2015020558.

[0014] In modern GPU architectures, many threads of execution execute concurrently, and each warp, which contains many threads, also executes concurrently. When threads in a warp need to perform more complex communications or collective operations, developers can use NVIDIA's CUDA "synchronize warps (_syncwarp)" primitive to synchronize the threads. The _syncwarp primitive initializes a hardware mechanism that causes the threads of execution to wait until all threads specified by a mask have called a primitive with the same mask before resuming execution. For more details, see, for example, U.S. Patents Nos. 8,381,203, 9,158,595, 9,442,755, 9,448,803, 1,000,2031, and 1,001,3290; see also https: / / devblogs.nvidia.com / using-cuda-warp-level-primitives / and https: / / docs.nvidia.com / cuda / cuda-c-programming-guide / index.html#memory-fence- functions .

[0015] While hardware-implemented barriers have proven useful, it is sometimes helpful for a program to use more than one barrier at a time. For example, a program could potentially use a first synchronization barrier to block a first set of threads and a second, different synchronization barrier to block another set of threads (or sometimes, the same synchronization barrier is reused to repeatedly block the same set of threads as their execution paths progress). In the past, to perform multiple barrier operations, software developers typically needed to indicate to the compiler in advance how many barriers were required. In systems where synchronization barriers are implemented in hardware, the number of available synchronization barriers is limited. Some programs require or may use more synchronization barriers than are supported in hardware by the hardware platform.

[0016] Due to other uses and needs for synchronization barriers, there is a need to improve the distribution of synchronization barriers. In particular, some past hardware-accelerated barrier implementations and methods have significant drawbacks:

[0017] 1. Programs that require more than one physical hardware barrier have difficulty allocating them.

[0018] 2. Barriers with a classic "arrive-and-wait" interface do not hide synchronization delays well (see Figures 1A-1H The "rabbit" thread may have to wait for a while and do nothing until the "turtle" thread catches up).

[0019] 3. Copy engines (such as direct memory access (DMA) units) generally cannot directly participate in hardware-based synchronization because they are not software threads.

[0020] It has long been possible to implement synchronization barriers in software, but software-implemented barriers do not necessarily provide the same level of performance as hardware-implemented barriers. For example, in the past, some developers used hardware to implement the barriers supported by the platform hardware and, if more (or different kinds of) barriers were needed, implemented the others in software. Developers who implemented synchronization barriers in software often suffered a performance penalty. In particular, over-allocating barriers can mean fewer execution paths and correspondingly reduced performance. Determining whether performance can be improved by using more barriers and fewer tasks is not always an easy trade-off for developers.

[0021] There is a need to improve the allocation of hardware acceleration and / or support for synchronization barriers in a manner that provides the flexibility of software allocation but does not adversely impact performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figures 1A-1H A time series of images is provided together, including a flipchart animation demonstrating the use of synchronization barriers (to view the animation, download the electronic copy of the patent from Figure 1A Start by repeatedly pressing the "Page Down" key);

[0023] Figure 2 An exemplary non-limiting instruction flow is shown;

[0024] Figure 2A and 2B is a block diagram of an exemplary non-limiting system employing the synchronization barrier technique of the present invention;

[0025] Figure 3 is a block diagram of an example non-limiting memory-backed synchronization barrier primitive;

[0026] Figure 4 It is available for management Figure 3 A block diagram of an example non-limiting combination of barrier hardware and software functionality.

[0027] Figures 5A-5F An example flow chart illustrating operations associated with a barrier function;

[0028] Figure 6 and 7 An illustrative two-thread barrier example is shown;

[0029] Figure 8 An illustrative example of a three-stage pipeline flow code is shown; and

[0030] Figure 9 An exemplary non-limiting cyclic code structure is shown.

[0031] Detailed Description of Example Non-Limiting Embodiments

[0032] We introduce a new type of barrier that solves the above problems:

[0033] 1. It is implemented in memory and hence allocated like memory.

[0034] 2. Split the "arrive" and "wait" operations so that unrelated tasks are executed in between.

[0035] 3. Asynchronous replication hardware from the same Streaming Multiprocessor (SM) can participate as a virtual or "implied" thread.

[0036] Implementing barriers in memory is perfectly feasible and is often done in software. Software split barriers, while less common, do exist in the field. We provide hardware acceleration for these idioms and integrate the hardware copy unit with the hardware acceleration, as if the hardware copy unit were a "memory" thread.

[0037] We have significantly improved the programming effort required to create feature-rich, performant barriers. By providing an innovative way to synchronize with copy operations, we can also introduce more asynchronous copy operations into the GPU's Streaming Multiprocessors (SMs), improving SM performance by offloading work from core threads.

[0038] Therefore, this example non-limiting technique provides additional functionality to barriers, which in turn can increase the popularity of synchronization barriers. In particular, more code than in the past may need (and be able to) use multiple synchronization barriers.

[0039] Split Arrive-Wait Barrier Synchronization Feature

[0040] As mentioned above, barrier synchronization is generally or typically defined as a construct in which some set of threads or other processes are blocked at a synchronization point, and when all threads or other processes in the specified set reach the synchronization point, then all threads or other processes are unblocked. Figures 1A-1H .

[0041] In particular, in many existing synchronization barrier implementations, such as Figure 1A Threads that arrive at the synchronization point early simply wait and do no useful work until they are unblocked by threads that complete later. A typical scenario is that as more and more threads arrive at the synchronization point, soon only one straggler thread remains unfinished, and all other threads wait before they all become unblocked and move on to the next stage of processing.

[0042] Suppose a programmer wants to implement a barrier in software. Assume the programmer / developer writes code to implement the barrier. An economical operation is "arrive and wait." This is how "Open MP" barriers are implemented, as well as previous CUDA barriers. One exemplary implementation would involve each thread that reaches a synchronization point declaring, "I've arrived." The program would then count the number of threads that have arrived. If not all threads have arrived, the program will block. Such a system would simply wait and spin, polling a counter until it reaches the correct value indicating that all threads have arrived. This polling loop is wasteful because threads aren't doing any useful work while waiting, and the processor spends a lot of time polling the counter, consuming resources that could be used to do useful work.

[0043] Some recent software-based approaches have decoupled the two events of arriving at a synchronization point and blocking at a synchronization point. The exemplary non-limiting embodiments herein similarly provide a decoupling of how threads or other processes arrive at and wait at synchronization points. In the exemplary non-limiting implementation, such techniques are used so that threads first arrive at a synchronization point that takes their arrival into account. However, in the exemplary non-limiting implementation, threads do not necessarily block upon arriving at a synchronization point. Instead, they can perform other work unrelated to the synchronization point (i.e., work that does not require synchronization with this particular synchronization barrier but is asynchronous with respect to this particular synchronization barrier). Once they have completed this other work and need to return to work that requires synchronization with the barrier, they can block if necessary. However, if the other work is sufficiently important, by the time a thread completes its other work, all other threads will have already reached the synchronization point, and no blocking will occur at all. In this case, the exemplary non-limiting implementation simply notes that all threads have reached the synchronization point and atomically blocks and unblocks any thread without actually delaying or preventing it from continuing its processing (unless the thread has exhausted other work it was performing that it might have done while waiting to unblock at the synchronization point).

[0044] The example non-limiting embodiment therefore separates the "reach and wait" function into two distinct atomic functions: (1) reach, and (2) wait. The "reach" portion of the function is all the bookkeeping and other housekeeping that would normally be done before implementing a barrier, but what it does not do is cause any threads to actually be blocked. Because threads are not blocked, they are allowed to perform work unrelated to the cause of the barrier.

[0045] For example, barriers are often used to implement various stages of computation on data structures. Thus, synchronization barriers are used to prevent any thread from using a data structure until all threads have completed their updates to the data structure. In an exemplary, non-limiting embodiment, a thread that has reached a synchronization point can be allowed to perform other useful work that does not involve the data structure while waiting for other threads to reach the synchronization point.

[0046] Example - Red and Green Data Structure

[0047] like Figure 2 In the example shown, we assume there are two data structures: Even though the patent drawings are not in color, for convenience, we will refer to two different data structures, which the system will update as a "red" data structure and a "green" data structure. Figure 2 It's more general than updating two different data structures (e.g., it also applies to updating a single data structure and then performing "other work" unrelated to that data structure while waiting for the update to that data structure to complete), but it's useful to illustrate how to update two different (e.g., "red" and "green") data structures using the procedure shown.

[0048] A "red" synchronization barrier is created (2100') to provide a barrier for the "red" data structure. Once the threads have completed updating the "red" data structure (2702) and have reached the "red" synchronization point (2200') after completing their respective operations on the "red" data structure, they can begin performing other work, such as work related to the "green" data structure (2704), and update that data structure while they wait for the "red" synchronization barrier - the work protecting the "red" data structure is completed. If protection of the "green" data structure is required, an additional "green" synchronization barrier can be similarly used.

[0049] Once the threads have completed their work on the "green" data structure, they can return to their work on the "red" data structure - but before doing any more operations on the "red" data structure, they need to ensure that the previous processing stage has completed. At this point, if the previous processing managed by the "red" synchronization primitive has not yet completed, the thread may need to wait (2300') until the processing stage is completed. Nevertheless, because the "reach" (2200') and "wait" (2300') atomic operations have been separated in time by an arbitrarily long amount of time, for example involving thousands of cycles, a lot of useful work (2704) can be performed collectively by any thread that has reached (2200') the "red" synchronization point but is not blocked and is instead free to perform any useful work other than on the "red" data structure.

[0050] It turns out that synchronization primitives never actually block any threads. If all threads are designed so that they begin working on the "green" data structure (2704) upon reaching the "red" synchronization point (2200'), and if, after working on the "red" data structure, the threads spend more time on the "green" data structure than the last stragglers to reach the synchronization point, then none of the threads will block. Instead, the synchronization primitives will transition the state to the next processing stage after the last stragglers reach the synchronization point, and when the threads check the state at the synchronization point, they will find that the processing stage has changed, so they don't need to block. Thus, no thread blocks and cycles are wasted.

[0051] Another way to describe this is that a synchronization barrier requires all threads or other processes to arrive at the synchronization point at the same time, and no thread or process is allowed to leave until all threads or processes have arrived. Instead of an "arrive and wait" scenario, the example non-limiting embodiment converts the "arrive" event into an execution window between "arrive" (2200') and "wait" (2300'). Threads are not allowed to pass "wait" (2300') until all threads have at least reached the "arrive" point (2200'). However, this does not prohibit threads that have already "arrived" from executing other tasks (2704) that are not protected by the synchronization barrier.

[0052] Once all threads and other processor processes reach a synchronization point, the exemplary non-limiting synchronization barrier herein is reset to begin the next processing phase. Thus, the synchronization barrier in the exemplary non-limiting embodiment is a multi-purpose object that can be used to manage multiple synchronization points for the same set of threads. Once the first phase is complete, the next phase managed by the same synchronization barrier can begin, which can then be followed by the next phase managed by the same synchronization barrier, and so on.

[0053] In more detail, in one exemplary, non-limiting embodiment, each thread participating in the reach-wait barrier calls two functions in sequence, first the wait (ARRIVE) function and then the wait (WAIT) function. The reach-wait-barrier model divides barrier participation in a program into three sections: the pre-arrival section (PRE_ARRIVAL_SECTION), the middle section (MIDDLE_SECTION), and the post-wait section (POST_WAIT_SECTION), with the following postulates:

[0054] PRE_ARRIVAL_SECTION ARRIVE[Arrive-Wait-Barrier Address] MIDDLE_SECTION WAIT[Arrive-Wait-Barrier Address] POST_WAIT_SECTION

[0055] Among them, in the example embodiment:

[0056] 1. Ensure that the thread's PRE_ARRIVAL_SECTION loads / stores are visible to the POST_WAIT_SECTION loads / stores of other participating threads;

[0057] 2. Ensure that a thread's POST_WAIT_SECTION loads / stores are not visible to other participating threads' PRE_ARRIVAL_SECTION loads / stores; and

[0058] 3. A thread's MIDDLE_SECTION load / store instructions have no visibility guarantees with respect to other threads.

[0059] In an example non-limiting embodiment, the reach-wait-barrier allows overlapping barriers to be superior for resolution.

[0060] It can be implemented as a software-implemented barrier, a hardware-implemented barrier, or a hybrid hardware / software-implemented barrier. For example, the previous CUDA hardware-based _synch primitive can be modified with circuit changes to implement the reach and wait strategy as two separate atomic functions (reach and wait) as described above. However, by implementing the synchronization barrier as a memory-backed, hardware-accelerated barrier, additional advantages are obtained in the exemplary non-limiting technique.

[0061] Memory-backed synchronization barriers

[0062] For the purpose of terminology clarification, the term "barrier" can mean different things depending on the level of abstraction. At lower levels of abstraction, a system typically has physical storage that implements stores. Functions that implement loads and stores are used to read and write to this storage.

[0063] At the next level of abstraction, when memory is used to pass data between processes, a mechanism can be provided to ensure that all relevant data has been written to physical memory before a flag is set to indicate to another process that the data is available for communication with the other process. Without some type of barrier, another process might attempt to read the data before or as it is written, and the message might be incomplete or incorrect. Such a barrier that prevents this is often called a "memory barrier."

[0064] In the context of the above example, memory barriers are not synchronization primitives, but rather side-effect-free instructions that are used to make operations visible to program instructions on machines such as many modern GPUs that reorder memory transactions. Many modern GPUs have such memory barriers, such as instructions called "memory fences" like CUDA's "__threadfence_block()" command. These memory barriers operate at a lower level of abstraction than typical synchronization barriers.

[0065] It's possible to implement synchronization primitives using memory barriers. Locks and mutexes (mutually excluded objects) are other examples of synchronization primitives. A mutex grants exclusive access to a critical resource. Barriers have similar functions, but with some differences.

[0066] Generally speaking, if Figure 1A As shown in

[15] , synchronization barriers separate different phases of computation. Many threads working in one phase reach the barrier, blocking them until all other threads involved in that phase complete their work. Once all threads reach the barrier, the barrier unblocks, and a new phase of processing begins. Such synchronization barriers are often defined as synchronization "primitives" and, in some cases, can be treated as objects. Thus, in some cases, synchronization primitives can be partially implemented using "memory barriers," but the discussion in this specification aims to separate the concept of "memory barriers" from that of "synchronization barriers."

[0067] The exemplary, non-limiting embodiment uses memory-backed synchronization barriers, i.e., uses memory storage (and, in some cases, associated memory barriers) to implement the synchronization barrier. By implementing the synchronization barrier as memory storage, the synchronization barrier is virtualized in the same way as virtualized memory. Furthermore, at least insofar as any virtual memory locations can be used to reinforce and support the synchronization barrier, there is no practical limit to the number of barrier objects that can be instantiated.

[0068] For example, suppose a synchronization barrier object consumes 64 bytes of storage. It follows that a storage-backed synchronization barrier scheme allows developers to have as many synchronization barriers as the available storage can accommodate the additional 64-byte long-term storage element. In modern GPU architectures with unified memory, global storage can be quite large, meaning many synchronization barriers can be accommodated. This is an improvement over hardware-backed synchronization barriers, which typically only have a limited number of barriers available, depending on the specific hardware implementation, chip design, available chip real estate, and so on.

[0069] By instantiating synchronization barrier objects in memory, the performance tradeoffs discussed above can be greatly simplified, because implementing objects in memory is a straightforward matter that most developers know how to do. Because developers can instantiate so many barrier objects (which, while not actually unlimited, is effectively unlimited as the size of main or global storage increases), there is no need to make a trade-off between the number of synchronization barriers and task capacity.

[0070] Because synchronization barriers are stored, i.e., implemented in memory in the example embodiment, they benefit from memory sharing and a hierarchical memory architecture. Previous hardware synchronization barrier circuitry was typically implemented directly within the processor. Therefore, any such hardware-implemented barriers typically did not cross between different processors. In other words, each processor could have its own hardware-based barriers that could be used to manage multithreaded tasks executing on that processor, but those hardware barriers would not be helpful for coordinating activities outside of a particular processor, for example, in a system where multiple processors might be involved in parallel implementations of the same processing stage. Such coordination typically required the use of shared global (CPU) main memory, which could be slow and have other performance issues.

[0071] In contrast, the exemplary, non-limiting embodiment of implementing synchronization barriers using memory instructions makes it possible to support functionality beyond the scope of a processor, GPU, or SOC (system on a chip). In particular, synchronization barriers can now be implemented at any level of a memory hierarchy (e.g., including levels shared across multiple cores, multiple streaming multiprocessors, multiple GPUs, multiple chips, multiple SOCs, or multiple systems) and, in some cases, cached in memory caches based on such a hierarchy.

[0072] Example non-restrictive memory supported system implementation

[0073] For example, reference Figure 2A An example system of the invention, which shows one or more CPUs 101, one or more GPUs 110 with local graphics memory 114, one or more main memories 115, and one or more displays 112, wherein synchronization barriers may be stored in the local graphics memory 114 and executed by all streaming multiprocessors (SMs) 204 in the GPU 110 (see Figure 2B ) or can be stored in main memory 115 and shared between CPU 101 and CPU 110. In more detail, local graphics memory 114 can be organized into different hierarchical structures, such as level 3 cache, level 2 cache, level 1 cache, shared memory, etc., all managed by memory management unit (MMU) 212, such as Figure 2B Since the storage hierarchy is used to determine which storage locations are shared between which resources, storing synchronization barriers at different levels of the storage hierarchy will control which computing resources can share the synchronization barriers (and similarly, which computing resources cannot access the barriers).

[0074] Thus, such memory-implemented synchronization barriers can be used to synchronize between threads running on a common core 206, between different warps running on different cores, between different processes running on the same or different GPUs 200, between the same or different SOCs, etc. Thus, a particular barrier is no longer limited to synchronizing threads processed in parallel on a particular processor, but can also be used to synchronize more threads or other executions across any number of different cores, GPUs, processors, processor architectures, chips, and systems.

[0075] This functionality can be enabled by implementing memory-backed synchronization barriers at the appropriate level of the memory hierarchy so that it can be accessed and shared by multiple processes and / or processors while being protected, for example, by common memory barriers. Furthermore, in terms of scalability, as the memory hierarchy grows, more and more threads can use these synchronization barriers, while smaller levels can support more limited barrier usage.

[0076] Using synchronization barriers to synchronize hardware

[0077] Another limitation of most existing hardware synchronization barrier implementations is that they can block software execution, but not necessarily hardware processes. One process that previous GPU hardware commonly performs is a copy operation. Although the processor can use memory commands such as load and store executed by the load / store unit 208 to copy from one memory location to another, such as Figure 2B The so-called "copy engine" 210 or "direct memory access (DMA) controller" (e.g., hardware accelerated or hardware-based copy operators) shown has long been used to accelerate the copying of data to and from memory. In many systems, software can send commands to a dedicated copy / DMA engine 210 that performs the data transfer. Such a copy engine can copy data for a thread from multiple consecutive locations in memory 214 corresponding to each offset from the thread's base address and store the data contiguously in system memory. The copy engine 210 can also copy the thread's data from a location in system memory and store the data in GPU memory 214. See, for example, U.S. Patent Publication No. 20140310484.

[0078] Typically, such a copy operation needs to be completed before the next processing stage begins because, for example, moving to the next processing stage may depend on the completion of updates to data structures in memory. Nevertheless, because previous implementations of synchronization barriers were based on blocking threads rather than hardware implementations, additional mechanisms are needed in addition to hardware-based synchronization primitives to ensure that the correct data exists after all threads monitored by the synchronization primitives complete before the next processing stage begins. In other words, in traditional approaches in the past, such additional hardware-based mechanisms (e.g., copy engine 210 or any other hardware) generally cannot participate in the same synchronization barrier process as the executing threads. Although a previous solution was to wrap software operators around hardware-based DMA / copy operators so that the software operators are completed only after the hardware operators complete, this approach imposes additional constraints on the software design that are not always satisfactory or effective.

[0079] In contrast to such prior approaches, one exemplary non-limiting feature integrates direct memory access (DMA) copy operations performed by the copy engine 210 (or other operations similar or performed by hardware such as a compute engine) into software-implemented but hardware-accelerated synchronization primitives so that the same synchronization primitives can be used to provide barriers for software processes, hardware-based processes, and hybrid hardware- and software-based processes.

[0080] Thus, one example non-limiting feature of embodiments herein is the utilization of synchronization barrier technology to synchronize transactions of the replication engine 210. For example, such integration could be performed using a purely hardware-based barrier implementation, but the example non-limiting memory-backed synchronization barrier technology described herein provides additional benefits in terms of performance and flexibility compared to a purely hardware implementation.

[0081] In one exemplary, non-limiting embodiment herein, once initiated, hardware operations such as the copy engine 210 operation behave as if they were fully-fledged threads from the perspective of a synchronization barrier, i.e., as if they were a stream of execution of software instructions that a programmer has written or a compiler has compiled. This implementation is elegant because it is simple to describe: the hardware operation behaves as if it were "morally" a single thread. In some exemplary, non-limiting embodiments, there can be many fine-grained hardware operations, such as copy operations, executing simultaneously and concurrently, and they can all be synchronized on a common synchronization barrier.

[0082] Using massively parallel modern GPUs, the most common approach to performing complex computations is collective. Consequently, computations can be performed collectively using a large number of threads, which can in turn collectively initiate an even larger number of hardware-based operations, such as DMA operations, through one or any number of copy engines 210. For example, assume 100 threads are executing simultaneously, and each of these 100 threads initiates a DMA operation by its associated copy engine 210. Using the example, non-limiting functionality of the techniques herein, the same synchronization barrier can synchronize 100 DMA operations and 100 threads (i.e., from the perspective of synchronization primitives, the DMA operations "look" like threads), providing synchronization for 200 processes (100 software threads and 100 hardware-based DMA operations). This functionality is provided, for example, by hardware acceleration circuitry that provides an interface between the MMU 212 and the copy engine 210, enabling the copy engine 210 to cause changes to the value of memory-backed synchronization primitives (e.g., incrementing and resetting counter values). The present technique is scalable, so any number of fine-grained DMA operations can be synchronized over the same barrier.

[0083] In a massively parallel architecture capable of supporting multiple threads, it may be inefficient to program each individual thread to wait for each hardware-based operation to complete. Instead, the present example non-limiting technology provides a synchronization primitive that allows a large number of threads (and in some embodiments, copy operations) to collectively wait for the completion of one or more hardware-based (e.g., copy) operations upon which the next processing stage depends.

[0084] In this context, the barrier primitive is a different type of mechanism than semaphores or flags (sometimes used in previous approaches to synchronize with hardware-based processes) because the new synchronization primitive provides collective synchronization. It is not the same as one thread setting a flag or semaphore for another thread. Instead, it allows N threads to block while potentially M hardware-based copy operations complete, where N and M are any non-negative integers. Such collective functionality is not necessarily limited to software-based or memory-backed barrier techniques, but can be implemented in software, hardware, or both.

[0085] Hardware-accelerated synchronization barriers

[0086] To achieve the above functionality and provide higher performance, the exemplary non-limiting embodiment provides a hardware-accelerated implementation of a memory-backed barrier. This implementation is invoked by software commands, but integrates other hardware functions (such as hardware-based copies) into the same synchronization barrier mechanism used to block software processes (such as threads).

[0087] While it is possible to implement such functionality entirely in software, in an exemplary, non-limiting implementation, hardware acceleration is used to more efficiently implement at least the barrier reset functionality and is also used to interact with hardware-based processes, such as the DMA copy engine, thereby allowing the hardware functionality to reset the barrier. In some embodiments, a dedicated hardware-based accelerator may be used to cache synchronization barriers.

[0088] In previous versions of software implementations, the last thread to arrive recognized that it was the last thread to arrive and modified the counter accordingly by adding the complement of the current value of the counter to reset the counter to the starting value. For example, see the Java "Phaser" typically implemented in the Java virtual machine. In some example implementations, the DMA engine is not written in software. Since the DMA engine may be responsible for resetting the barrier in some cases, and since they are not software, the operation of performing such a reset in these implementations is desirable to be performed in hardware. For this reason, the example non-limiting embodiments provide a hardware-accelerated reset operation. However, other example non-limiting implementations of the technology herein can be applied to phasers, latches, or other synchronization primitives other than barriers. Such techniques can also be used for semaphores.

[0089] Software implementation

[0090] In the exemplary non-limiting embodiment, each reach-wait-barrier state is an implementation-defined (e.g., 64-bit) data structure 1900 stored in memory. Figure 3 As shown, the data structure 1900 has the following contents:

[0091] 1. Expected number of arrivals per use of the barrier (field 1908).

[0092] 2. The remaining number of arrivals required to clear the barrier (counter field 1904)

[0093] 3. Barrier phase (for barrier reuse) (field 1902).

[0094] The reach-wait-barrier in the exemplary non-limiting embodiment allows threads to cooperate, so the counts in fields 1904, 1908 may be represented as thread counts.

[0095] like Figure 3As shown, the first field 1902 is a stage counter that indicates the current processing stage of the reach-wait barrier. After resetting the barrier, the stage counter 1902 can be incremented to the next stage number. In one example implementation, the stage counter 1902 can be a single bit status flag (i.e., a one-bit counter) that is flipped (incremented) each time the barrier is reset. Such a status flag can be used to indicate to the thread that the state of the barrier has changed to the next processing stage - meaning that the thread is no longer (or no longer needs to) block the barrier. In other example non-limiting embodiments, additional resolution of the stage indicator may be required to distinguish between the current stage and the next consecutive stage. Such other implementations may, for example, increment a multi-bit count each time the barrier is reset to track which stage of N processing stages is currently being processed, where N is an integer.

[0096] like Figure 3 As further shown, additional fields include an arrival counter 1904, which indicates how many threads / processes have arrived (or, in one implementation, the remaining arrivals required to clear the barrier). In one example implementation, the arrival counter 1904 is incremented each time a thread or other process reaches the barrier. When the arrival counter 1904 increments to a predetermined known value, this indicates that all threads and processes have arrived, and the barrier can change state and move to the next processing stage.

[0097] When the last stragglers (or hardware processes) arrive and the barrier is satisfied / reset, the arrival counter 1904 may be reset, e.g., Figure 3 The exemplary, non-limiting embodiment shown contains an initial value in the third field 1908 - namely, the expected number of arrivals each time the arrive-wait barrier is used.

[0098] As discussed below, the exemplary non-limiting embodiment allows software to dynamically change the values ​​of the arrival counter 1904 and the "expected number of threads" field 1908 after the barrier is created.

[0099] In an exemplary non-limiting embodiment, Figure 3The data structure 1900 shown can be stored anywhere in memory. As described above, the type of memory / hierarchical level of memory used to store the data structure 1900 is selected to provide the expected scope of sharing between threads. When a thread implements the synchronization barrier of the example non-limiting embodiment of the present invention, the synchronization primitive call in the thread directly or indirectly includes a memory address for storing an instance of the data structure 1900 representing the synchronization primitive. This is different from existing hardware-based synchronization primitives, in which the synchronization primitive call may list a reference number or ID. The memory address (which can be a physical or virtual memory address depending on the system) indicates where in memory the instance of the synchronization primitive 1900 is to be found by the thread.

[0100] Example hardware acceleration implementation

[0101] Figure 4 yes Figure 3Block diagram of an example, non-limiting, hardware-accelerated memory-backed implementation of synchronization primitive 1900. As described above, current stage counter 1902, arrival counter 1904, and expected number storage field 1908 are stored in memory and accessible by load / store unit 208, which, in an example embodiment, is capable of performing atomic operations (stored in processor instruction register 1952 and decoded by conventional instruction decoder 1954), such as (arithmetic functions, e.g., atomicAdd(), atomicSub(), atomicExch(), atomicMin(), atomicMax(), atomicInc(), atomicDec(), atomicCAS()); bitwise functions, e.g., atomicAnd(), atomicOr(), atomicXor()); and other functions. Such atomic operations allow a streaming multiprocessor to change the value of synchronization primitive 1900 "in place." The MMU 212 is modified to include hardware circuitry that allows the DMA controller / copy engine 210 to similarly modify the arrival counter 1904 "in place" and reset the synchronization primitive 1900 (which operates as a comparator that compares the count in the counter to a predetermined value and resets the counter and phase indicator based on the comparison result) if the copy engine operation causes the arrival counter decoder 1956 to determine that there is no longer a thread or copy engine operation waiting before resetting the synchronization barrier. In this case, the decoder 1956 initiates a hardware-controlled reset of the synchronization primitive 1900 instance to "flip" the phase indicator 1902 and reload the expected number 1908 into the arrival counter 1904. In an example embodiment, the LSU 208 and / or MMU 212 circuitry performs a direct memory access atomic operation to initiate / perform the appropriate atomic operation on the synchronization primitive 1900 instance stored in memory to make these changes.

[0102] Therefore, in an exemplary non-limiting embodiment, the processor architecture is modified to provide additional circuitry for binding hardware-based processes (e.g., DMA) into the synchronization barrier implementation, enabling the hardware to reset the synchronization barrier after completing its DMA task and recognizing that it is the last stragglers required to proceed to the next stage of processing. In the exemplary non-limiting implementation, as described above, the hardware modifications need not involve separating arrivals by waiting, since typically, once the hardware DMA controller completes the previous task, it will return and prepare for the next task. Nonetheless, in the exemplary non-limiting implementation, the count maintained by the barrier's completion counter 1904 will include the number of threads required to complete and the number of DMA hardware operations required to complete - that is, the counts 1904, 1908 will not distinguish between thread counts and DMA / copy engine counts, but each count will contain a value that summarizes the number of threads and copy operations that must be completed to reset the thread.

[0103] Exemplary non-limiting implementations

[0104] The example non-limiting embodiments herein implement changes to the instruction set architecture (ISA) to include instructions for accessing new synchronization primitives that reference a synchronization barrier stored in memory backed by a memory. Furthermore, in the example non-limiting implementation, instead of one primitive call, a thread will include two different primitive calls: a "reach" primitive call to the barrier and a "wait" primitive call to the barrier. Between these two calls, as described above and as Figure 2 As shown, a thread can contain instructions unrelated to the barrier that can be executed without violating the barrier.

[0105] During initialization, the system will first set up a synchronization barrier instance in memory and store the appropriate data the system needs to retain to implement the barrier (e.g., arrival count, phase count). Typically, the SDK (software development kit) provided by the system designer may include a library containing these various function calls to initiate the synchronization barrier. Similarly, the processing system's ISA is modified to include new instructions for synchronization barrier arrival and synchronization barrier wait.

[0106] In one example, non-limiting embodiment, the following software functions may be used to manage the reach-wait-barrier primitive 1900 instances stored in memory:

[0107] Create (_create) function 2100 (see Figure 5A ) is used to set up a reach-wait-barrier in memory.

[0108] Arrival (_arrive) function 2200 (see Figure 5B) is used by a thread to indicate its arrival at the wait-barrier. This function returns the barrier stage for use in the wait (_wait) function.

[0109] Wait function 2300 (see Figure 5C ) is used to wait for a reach-wait-barrier to clear the provided stage.

[0110] ·Terminate thread (_dropthread) function 2400 (see Figure 5D ) Permanently removes the thread from the reach-wait-barrier. This is useful when the thread exits.

[0111] · Increment (_addonce) function 2500 (see Figure 5E ) Increases the number of reach-wait-barriers by a specified amount.

[0112] · Decrement (_droponce) function 2600 (see Figure 5F ) reduces the number of reaches-wait-barriers by a specified amount.

[0113] Furthermore, some non-limiting embodiments include an additional instruction ARRIVES.LDGSTSBAR.64 that signals that all DMA transfers from this thread are complete and updates the arrival count in the arrive-wait-barrier accordingly.

[0114] _create

[0115] Figure 5A An example _create function 2100 is shown that sets a new barrier. In this example, the calling code is responsible for calling _create in only one thread. The _create function 2100 takes as parameters "Ptr" (the direct or indirect storage location in memory where the barrier data structure is to be stored) and "ExpectedCount" (the expected set of total number of threads and DMA operations to be synchronized by this particular barrier). In the example shown, a counter in the data structure is initialized (2102). The barrier counter is set to ExpectedCount (incremented when it arrives) (2108), and the now initialized barrier is stored to the specified location in memory (2110). The function then returns (2112). As described above, in one example embodiment, the barrier can be stored to shared memory, but in other embodiments, it can be stored to any desired memory in the storage hierarchy consistent with the synchronization scope.

[0116] A more specific example is:

[0117] _create(BarPtr,NumThreads)

[0118] Input: BarPtr, NumThreads

[0119] Initializes the barrier for the specified number of threads.

[0120] Core primitives: Cooperative Thread Array (CTA) wide split barrier (dispatched via CTA wide synchronization)

[0121] BarPtr = pointer to the allocated barrier stored in a shared or any other memory location that stores opaque barrier state

[0122] NumThreads: The number of threads participating in this barrier that need to arrive before the wait is cleared.

[0123] _arrive

[0124] Figure 5B An example _arrive function 2200 is shown that indicates that a thread has reached a barrier and returns the barrier stage for use in a _wait() function 2300. In the example shown, the _arrive function 2200 reads the stage of the barrier in memory location [Ptr] and stores it in a local stage (e.g., in a register accessible to the thread). Note that in the exemplary non-limiting embodiment, from a software perspective, the _arrive function 2200 does not change the state of the barrier, but only reads the current stage value from the barrier. Nonetheless, in the example implementation, calling the _arrive function 2200 will have the effect (based on hardware acceleration associated with the barrier implementation) of decrementing the number of threads indicated by the barrier's thread counter, thereby "registering" with the barrier that the thread has reached a synchronization point, so that the barrier no longer waits for that particular thread to reach its defined synchronization point.

[0125] In an exemplary non-limiting embodiment, the _arrive function 2200 call can be placed anywhere in a thread, and the location of the _arrive function 2200 call defines a synchronization point within the thread. It is up to the developer and / or optimizing compiler to ensure that the number of threads containing the _arrive function call 2200 (+DMA or other appropriate hardware calls) matches the expected number of arrivals programmed into the barrier.

[0126] More specific non-limiting examples:

[0127] _arrive(BarPhase,BarPtr);

[0128] Input: BarPtr

[0129] Output: BarPhase

[0130] Indicates that the thread has arrived; return to the barrier stage used to wait for instructions.

[0131] In an example embodiment, a wait instruction may be initiated by the copy engine 210 independently of any executing software thread. This may be accomplished by hardware in the MMU 212 or LSU 209 generating a fused atomic load / store command (LDGSTS) to shared memory, which in essence is a direct memory access ("DMA") performed by the hardware engine to primitive entities stored in the shared memory.

[0132] _wait(BarPtr,BarPhase)

[0133] like Figure 5C As shown, the _wait function 2300 waits for the stage provided by the arrival-wait-barrier to clear. The _wait function 2300 again reads the stage of the barrier at memory location [Ptr] and compares it with the stage previously read by the _arrive function 2200 (2304). Therefore, Figure 5C An infinite loop is defined that waits until the barrier phase changes before allowing the thread to continue. A thread executing the _wait function 2300 will be blocked on the barrier until the barrier phase changes.

[0134] _wait(BarPtr, BarPhase) requires the use of a BarPhase returned by a previous call to _arrive. In the non-limiting example embodiment where _arrive produces side effects, _wait must also produce those side effects. However, more commonly, the opposite is true—that is, if _wait has side effects, _arrive must also have the same side effects.

[0135] In the particular example shown, since the _wait function 2300 uses the value retrieved by the _arrive function 2200, _wait 2300 should only be called after _arrive 2200 has been called. The two functions can be called immediately after each other, or any number of instructions or other functions not related to the barrier can be placed between the _arrive function 2200 call and the _wait function 2300 call. Developers (and / or optimizing compilers) may want to place useful work between the _arrive function call 2200 and the wait function call 2300 to avoid wasting processor cycles. A thread that calls the _wait function 2300 after the barrier stage state has changed will not block on the barrier, but will execute the next instruction after the function call, while executing Figure 5C Operations 2302, 2304 involve only a short time delay (eg, one or two cycles).

[0136] A more specific example:

[0137] _wait(BarPtr,BarPhase)

[0138] Input: BarPhase;

[0139] Waits until all expected arrivals for the specified phase of the barrier have occurred.

[0140] One thread for a barrier BarPtr

[0141] Each Wait(BarPtr) has a corresponding Arrive(BarPtr), and the BarPhase in Arrive(BarPtr) is provided as input to _wait(BarPtr). A call to _wait(BarPtr) cannot be followed by _wait(BarPtr) without an intermediate _arrive(BarPtr).

[0142] A call to _arrive(BarPtr) should not follow _arrive(BarPtr) without an intervening _wait(BarPtr).

[0143] _dropthread

[0144] In an example embodiment, using Figure 5D The _dropthread function 2400 is used to permanently remove a thread from the reach-wait-barrier. This function is useful when a thread needs to exit. The function's function is to decrement (decrement) the expected number of arrivals in the barrier so that the barrier will no longer wait for this particular thread (block 2402). In the specific example shown, this operation increments the value stored in the barrier, thereby reducing the number of counts required to reach the "wait for more threads" count.

[0145] A more specific example:

[0146] _dropThread(BarPtr)

[0147] Input: BarPtr

[0148] Removes a thread from a barrier. This is useful when the thread is waiting to exit.

[0149] _addonce(BarPtr,Count)

[0150] Figure 5E The _addonce function 2500 increases the count of the reach-wait-barrier by a specific amount, and Figure 5FThe _droponce function 2600 decrements the reach-wait-barrier count by a specific amount. Any overflow can be handled explicitly by software.

[0151] A more specific example:

[0152] _add(BarPtr,AddCnt)

[0153] Input: BarPtr, AddCnt

[0154] Adds additional expected arrivals for this barrier using AddCnt. This is used only once for this barrier.

[0155] For all threads participating in the barrier:

[0156] The sum of all AddCnt matches the number of _arrive() calls.

[0157] The thread providing _addCnt may be different from the thread executing _arrive()

[0158] A thread should not execute _add(BarPtr) between _arrive(BarPtr) and _wait(BarPtr)

[0159] Other ISA approaches are also possible. The main difference lies in how the expected barrier arrival count is specified. Some options include:

[0160] Specify expected arrivals at barrier creation

[0161] Specify expected arrival whenever a barrier is used by "arrival" (like existing SM CTA barriers, all threads must specify the same expected arrival)

[0162] Specify an expected arrival whenever the barrier is used by "wait" (like existing SM CTA barriers, all threads must specify the same expected arrival)

[0163] Hybrid: specify expected arrivals at barrier creation, but add additional expected arrivals via "arrivals" Example non-limiting microarchitecture (can be used to implement Figure 4 Block diagram of

[0164] Barrier state

[0165] *phase: The phase of the barrier

[0166] *count: barrier count

[0167] *ThreadCnt: The number of threads participating in this barrier

[0168] _create(BarPtr,InitCnt)

[0169] barrier[BarPtr].initCnt=initCnt;

[0170] barrier[BarPtr].cnt=initCnt;

[0171] barrier[BarPtr].phase=0;

[0172] _add(BarPtr,addCnt)

[0173] / / increment arrival count

[0174] barrier[BarPtr].cnt+=addCnt;

[0175] Arrive_function(BarPtr) / / this is not part of the API in oneembodiment

[0176] / / decrement arrival count

[0177] barrier[BarPtr].cnt--;

[0178] / / check if arrival is clearing barrier

[0179] if(barrier[BarPtr].cnt==0){

[0180] / / update phase and reset count

[0181] if(barrier[BarPtr].phase==0){barrier[BarPtr].phase=1;}else{barrier[BarPtr].phase=0;}

[0182] barrier[BarPtr].cnt=barrier[BarPtr].init;

[0183] / / unstall all warps waiting on barrier

[0184] unstall(BarPtr);

[0185] }

[0186] Arrive(BarPtr,addCnt,BarPhase)

[0187] / / return phase(optional)

[0188] BarPhase=barrier[BarPtr].phase;

[0189] / / Do arrive function

[0190] Arrive_function(BarPtr);

[0191] LDGSTS_Arrive(BarPtr)

[0192] / / Do arrive function

[0193] Arrive_function(BarPtr);

[0194] BarWait(BarPtr,BarPhase)

[0195] / / if barrier is in same phase,it has not yet cleared then stall,otherwise keep going.

[0196] if(barrier[BarPtr].phase==BarPhase){stall(BarPtr);}

[0197] DropThread(BarPtr)

[0198] / / thread is removing itself from the barrier

[0199] barrier[BarPtr].initCnt--;

[0200] / / Do arrive function

[0201] Arrive(BarPtr);

[0202] Cooperative data movement without thread synchronization. Split barrier based DMA tasks

[0203] The programming model may be one that matches that of a multithreaded barrier, except that the barrier is separate.

[0204] The existing multithreading barriers can be described as:

[0205] · <pre>

[0206] ·BARRIER

[0207] · <post>

[0208] Visibility rules:

[0209] Ensure thread <pre>Load / store operations on other participating threads <post>Load / store visible.

[0210] Ensure thread <post>Load / store operations on other participating threads <pre>Loads / stores are not visible.

[0211] The separate multithreading barrier can be described as:

[0212] · <pre>

[0213] ·ARRIVE

[0214] · <middle>

[0215] ·WAIT

[0216] · <post>

[0217] Visibility rules (the first two are the same as above)

[0218] Ensure thread <pre>Load / store operations on other participating threads <post>Load / store visible.

[0219] Ensure thread <post>Load / store operations on other participating threads <pre>Loads / stores are not visible.

[0220] Thread <middle>Loads / stores have no visibility guaranteed ordering with respect to other threads (at least not any visibility guarantees provided by that particular barrier)

[0221] Logically think of the LDGSTS "DMA" instruction as a separate thread that is "forked" by the caller thread and executes the LDG / STS / ARRIVE, after which it "dies".

[0222] Visibility issues with split barriers

[0223] Split barriers imply that multiple split barriers can overlap. All of the following overlaps are permitted and functionally correct without deadlock (see also Figure 6 ).

[0224] Pipeline

[0225] Thread 0 Thread 1 Arrive(BarA) Arrive(BarB) Arrive(BarB) Arrive(BarB) Wait(BarA) Wait(BarA) Wait(BarB) Wait(BarB)

[0226] Built-in

[0227] Thread 0 Thread 1 Arrive(BarA) Arrive(BarA) Arrive(BarB) Arrive(BarB) Wait(BarB) Wait(BarB) Wait(BarA) Wait(BarA)

[0228] Different instructions for each thread

[0229] Thread 0 Thread 1 Arrive(BarA) Arrive(BarB) Arrive(BarB) Arrive(BarA) Wait(BarA) Wait(BarB) Wait(BarB) Wait(BarA)

[0230] Different barriers and instructions per thread

[0231] Thread 0 Thread 1 Thread 2 Arrive(BarA) Arrive(BarC) Arrive(BarB) Arrive(BarB) Arrive(BarA) Arrive(BarC) Wait(BarA) Wait(BarA) Wait(BarB) Wait(BarB) Wait(BarC) Wait(BarC)

[0232] In some exemplary non-limiting embodiments, the following overlaps should not be allowed because they would create deadlocks.

[0233] →Deadlock

[0234]

[0235]

[0236] Status Example

[0237] Figure 6 An example of a sample state is shown. Figure 6 First, the transition from phase 0 to phase 1 is shown (when all transitions have occurred, the barrier phase is changed and the count is re-incremented). Thread 1's "Wait(BarPhase_T1, BarPtr)" operation ensures that all loads / stores from phase 0 by all participating threads are visible to thread 1 before continuing. Similar "Wait" operations in phases 1 and 2 provide similar functionality for these phases. Each time a barrier is used, it is a new phase, so in the example embodiment, the internal phase indicator only needs to be one bit to indicate which phase a particular Wait() call needs to wait on.

[0238] Figure 7 Another state example is shown, which includes load / store operations performed by a hardware operator, such as copy engine 210. The "Add(BarPtr, 2)" commands executed by thread 0 and thread 1 are used to add these two copy engine operations to the barrier.

[0239] In this example, the operation "Wait(BarPhase_T1, BarPtr)" ensures that thread 1 can see all loads / stores in stage 0 of all participating threads (marked by Arrive) and that all LDGSTS results are visible in shared memory (marked by LDGSTS Arrive). Once all arrivals have occurred (see the arrow lines indicating stage = 0 and then stage = 1), the barrier stage will change and the count will be recalculated.

[0240] Figure 8 An illustrative example of a three-stage pipeline streaming code is shown. Figure 8 The initiation of the loop is shown, followed by the iterations of the loop. Figure 9 An example of a loop code structure is shown with the following details:

[0241] ---URF / RF value---

[0242] URF1 / URF2: points to the barrier to be used

[0243]

[0244] …repeat…

[0245] All documents cited herein are incorporated by reference as if expressly set forth.

[0246] While the invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, the invention is intended to cover various embodiments and modifications and equivalent arrangements included within the spirit and scope of the appended claims.< / middle> < / pre> < / post> < / post> < / pre> < / post> < / middle> < / pre> < / pre> < / post> < / post> < / pre> < / post> < / pre>

Claims

1. A synchronization barrier, comprising: a data structure stored in the memory, the data structure comprising a counter; the counter being advanced by a first operation performed by an execution thread, and the counter being further advanced by a second operation performed by a hardware-controlled operator, the second operation performed by the hardware-controlled operator being capable of advancing the counter independently of the first operation performed by the execution thread, The data structure stored in the memory includes an arrival counter and another value for reinitializing the arrival counter upon a reset of the synchronization barrier. 2 . The synchronization barrier of claim 1 , wherein the data structure stored in the memory includes a phase flag. 3 . The synchronization barrier of claim 1 , wherein the first operation comprises reaching an ARRIVE different from waiting for WAIT and / or waiting for WAIT different from reaching an ARRIVE.

4. The synchronization barrier of claim 1 , wherein the data structure is configured to be reset in response to a fused load / store atomic that can be initiated by a hardware engine or a software thread. 5 . The synchronization barrier of claim 1 , further comprising circuitry associated with the counter, the circuitry enabling further execution of the same or a different thread of execution to be gated based on a state of the counter.

6. A computer system comprising: a synchronization barrier primitive stored in a memory, the primitive comprising a counter and a phase indicator, wherein the counter is associated with a set of threads and at least one copy operation performed by a hardware-controlled operator; as well as Memory access circuitry resets the counter and changes the phase indicator in response to the counter indicating that all threads in the set of threads and the at least one copy operation have reached a synchronization point and all operations in the set of threads have completed. 7 . The system of claim 6 , wherein the counter counts a total of the number of copy operations completed and the number of execution thread arrival calls.

8. The system of claim 6, wherein the memory access circuitry resets the counter and changes the phase indicator in response to execution of an instruction by a software thread, the instruction comprising arriving at an ARRIVE operation without waiting for a WAIT operation or waiting for a WAIT operation without arriving at an ARRIVE operation.

9. The system of claim 6 , wherein the primitives stored in the memory further include a predetermined value, and hardware resets the counter by loading the predetermined value when the counter indicates that all threads and copy operations in a thread set have reached a synchronization point and the copy operations in the set have completed.

10. The system according to claim 9, wherein the system is configured to allow a thread to dynamically change the predetermined value. 11 . The system of claim 6 , wherein the synchronization barrier primitive is stored in a shared memory of a graphics processing unit (GPU).

12. The system of claim 6, wherein the synchronization barrier primitives are stored in a storage hierarchy that determines access to the primitives by threads. 13 . The system of claim 6 , further comprising a comparator that compares the count of the counter with a predetermined value and resets the primitive according to a result of the comparison.

14. The system of claim 6, wherein the phase indicator of the primitive is configured to be read by first arriving at an ARRIVE command and then by waiting at a WAIT command, so that a thread can determine whether the phase indicator of the primitive has changed phase status.

15. A non-transitory readable medium storing a graphics processing unit (GPU) instruction set architecture, comprising: an arrival ARRIVE operation that reads at least a phase indicator portion of a synchronization barrier primitive stored in memory and causes the barrier primitive to advance a counter via a first operation performed by an execution thread and to advance the counter via a second operation performed by a hardware-based control operator; a wait WAIT operation that reads at least the phase indicator portion of a synchronization barrier primitive stored in the memory and compares the phase indicator portion read by the arrive ARRIVE operation with the phase indicator portion of the primitive read by the wait WAIT operation to determine whether the phase state of the barrier has changed; as well as A fused load / store instruction allows a hardware-based engine to reset the synchronization barrier primitive when the hardware-based engine completes a task assigned to the hardware-based engine.

16. The non-transitory readable medium of claim 15, wherein the GPU instruction set architecture further comprises an add (ADD) operation that adds a stored field to the synchronization barrier primitive, the field being used to reinitialize the primitive based on a reset of a next stage state. 17 . The non-transitory readable medium of claim 15 , wherein the GPU instruction set architecture further comprises a CREATE instruction that initializes the synchronization barrier primitive and stores the synchronization barrier primitive to a memory.

18. A synchronization method, comprising: storing in a memory a synchronization barrier tag comprising a phase indicator and a counter associated with a thread group and a task executed by a hardware-initiated controller; executing an arriving instruction using at least one thread in the thread group, thereby causing the counter to count and causing the thread to read the stage indicator; Using a hardware-activated controller to complete the task, thereby causing the counter to count; When the counter count indicates that the thread group has executed the arrival instruction and the hardware-enabled controller has completed the task, resetting the counter; as well as Executing a wait instruction using the at least one thread causes the at least one thread to read the stage indicator again, the at least one thread adjusting blocking based on whether the stage indicator has changed value.

19. The synchronization method according to claim 18, comprising: From when the at least one thread executes the reach instruction to when the at least one thread executes the wait instruction to open an execution window, the at least one thread executes work asynchronously with respect to a synchronization barrier within the execution window.

20. A synchronization barrier comprising: providing a counter for counting synchronization barriers, wherein the counter resides in a memory; as well as Circuitry is operatively connected to the counter, the circuitry advancing the synchronization barrier count in response to completion of a software-initiated operation performed by a thread of execution and in response to completion of an operation performed by a hardware-initiated operator, wherein the circuitry is configured to reset the synchronization barrier count in response to a fused load / store atomic that can be initiated by each of the replicating hardware and software thread execution.

21. The synchronization barrier of claim 20, wherein the memory comprises memory shared by an executing thread and the hardware-enabled controller.

22. The synchronization barrier of claim 20, wherein the counter is stored in a shared memory of a graphics processing unit (GPU).

Images