Atomic operation implementation method and device based on calculation in SRAM (Static Random Access Memory) cache
By performing atomic operations in the SRAM cache and managing concurrent access using spin locks, the problem of atomic operations performance loss in GPGPU is solved, efficient shared variable updates and synchronization are achieved, and the system's parallel processing capabilities and overall performance are improved.
Patent Information
- Application Number
- CN202410941341.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-12
AI Technical Summary
In GPGPUs, atomic operations result in performance losses and resource waste due to slow memory access and strict storage consistency requirements, especially in data transfer and synchronization operations across storage hierarchies.
By performing atomic operations in the SRAM cache, using spin lock and parallel access methods, atomic parallel processing in the cache is realized, data synchronization overhead and performance losses are reduced, and atomic cache design is used to manage the update and synchronization of shared variables to avoid thread switching The overhead of the context.
Improve the execution efficiency of atomic operations, reduce data transmission across storage hierarchies, relax the execution order of atomic operations, and maintain the semantic conditions of sequential consistency, obtain performance gains for removing memory barriers, and reduce synchronization overhead and Performance loss.
Smart Images

Figure CN118939573A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of computer architecture, and in particular to a method and device for implementing atomic operations based on calculations within an SRAM cache. Background Art
[0002] Atomic operations are a crucial mechanism for ensuring data consistency and avoiding race conditions. In general-purpose graphics processing units (GPGPUs), atomic operations are often used for data protection and synchronization when multiple threads or cores simultaneously access shared memory. In the von Neumann architecture, there's a speed disparity between memory and the GPGPU, with memory access being slower and the processor processing being faster. For atomic operations that require frequent memory access, the slow memory access speed and the need to traverse multiple memory hierarchies mean that the GPGPU wastes significant time waiting for memory data transfers, limiting overall system performance. Furthermore, atomic operations in GPGPUs must also adhere to a strict memory consistency model. However, these strict memory consistency requirements and the use of barrier synchronization can lead to performance penalties. Barriers block thread execution, preventing the thread from proceeding to the next step until all threads reach the synchronization point, resulting in additional waiting time and wasted resources.
[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention
[0004] The main purpose of the embodiments of the present application is to propose a method and device for implementing atomic operations based on calculations within the SRAM cache, which can reduce data transmission across storage hierarchies and perform parallel processing of atomic operations within the cache, thereby reducing data synchronization overhead and performance loss and improving the execution efficiency of atomic operations.
[0005] To achieve the above objectives, an embodiment of the present application provides a method for implementing atomic operations based on calculations within an SRAM cache, the method comprising:
[0006] Get atomic operation request instruction;
[0007] Based on the spin lock mode and the parallel access mode, according to the atomic operation request instruction, the atomic cache behavior and the atomic cache memory access behavior are executed to realize the atomic operation in the cache.
[0008] In some embodiments, the atomic cache behavior includes:
[0009] Obtaining the atomic operation request instruction and sending it to the atomic buffer, wherein the atomic operation request instruction is an atomic read-modify-write operation request instruction;
[0010] Determine the state of the memory access address of the atomic operation request instruction and output a memory access address state determination result;
[0011] An atomic read-modify-write operation is performed according to the memory access address state judgment result.
[0012] In some embodiments, performing an atomic read-modify-write operation according to the memory access address state determination result includes:
[0013] If the memory access address state judgment result is a memory access address locked state, resending the atomic operation request instruction after a preset static time;
[0014] If the memory access address status judgment result is that the memory access address is stateless, the atomic read-modify-write operation is performed according to the atomic operation request instruction, and the memory access address stateless is modified to the memory access address locked state, until the atomic read-modify-write operation is completed, and the memory access address locked state is modified to the memory access address stateless.
[0015] In some embodiments, the atomic cache access behavior includes:
[0016] Obtaining the atomic operation request instruction and sending it to the atomic buffer, wherein the atomic operation request instruction is an atomic read-modify-write operation request instruction;
[0017] Perform cache tag judgment according to the atomic operation request instruction, and output the cache judgment result;
[0018] The atomic read-modify-write operation is performed according to the cache determination result.
[0019] In some embodiments, performing the atomic read-modify-write operation according to the cache determination result includes:
[0020] If the cache determination result indicates a cache miss, a memory access request is initiated to the shared global memory, and a memory access request reply is obtained;
[0021] Sending the memory access request reply to the atomic cache to establish a cache tag and perform the atomic read-modify-write operation;
[0022] If the cache determination result indicates a cache hit, the atomic read-modify-write operation is performed directly.
[0023] In some embodiments, further comprising:
[0024] If the first atomic operation request instruction and the second atomic operation request instruction have the same cache tag, and when the cache determination result indicates a cache hit, the first atomic operation request instruction and the second atomic operation request instruction are executed in parallel.
[0025] To achieve the above objectives, another aspect of an embodiment of the present application provides an atomic operation implementation device based on SRAM cache calculation, the device comprising a thread block scheduler, several streaming multiprocessors, an L2 cache, and an atomic cache, wherein the output ends of the thread block scheduler are connected to the input ends of several streaming multiprocessors, the first input end of the L2 cache, and the first input end of the atomic cache, and the output ends of the several streaming multiprocessors are connected to the second input end of the L2 cache and the second input end of the atomic cache, wherein:
[0026] The thread block scheduler is used to control the scheduling and allocation of thread execution order and generate thread execution instructions;
[0027] The plurality of streaming multiprocessors are used to execute the thread execution instructions and coordinate data synchronization and sharing among threads;
[0028] The L2 cache is used to cache data for executing the thread execution instruction;
[0029] The atomic buffer is used to manage atomic operations for updating and synchronizing shared variables.
[0030] In some embodiments, the streaming multiprocessor includes an instruction cache and a data cache, wherein:
[0031] The instruction cache is used to accelerate the access operation of the thread execution instruction;
[0032] The data buffer is used to accelerate the access operation of data between the coordination threads.
[0033] In some embodiments, the streaming multiprocessor further includes a shared global memory, and the shared global memory is used for data interaction between the streaming multiprocessor and the L2 cache and the atomic cache.
[0034] In some embodiments, the atomic buffer includes an on-site storage type 8T SRAM array, a switch control circuit, a near-memory calculation logic, a data drive control circuit, and an SRAM control circuit, wherein the output end of the switch control circuit is connected to the first input end of the on-site storage type 8T SRAM array, the output end of the near-memory calculation logic is connected to the input end of the data drive control circuit, the output end of the data drive control circuit is connected to the second input end of the on-site storage type 8T SRAM array, and the output end of the SRAM control circuit is connected to the third input end of the on-site storage type 8T SRAM array, wherein:
[0035] The field storage type 8T SRAM array is used to store data generated by in-memory atomic operations and control the calculation of the in-memory atomic operations;
[0036] The switch control circuit is used to control the field storage type 8T SRAM array to switch between the SRAM mode and the atomic operation mode;
[0037] The near memory calculation logic is used to control the carry calculation of the in-memory atomic operation;
[0038] The data drive control circuit is used to control a number of the atomic operations in the memory;
[0039] The SRAM control circuit is used to control data access of the field storage type 8T SRAM array.
[0040] The embodiments of the present application include at least the following beneficial effects: The present application provides a method and device for implementing atomic operations based on calculations within the SRAM cache. The scheme obtains atomic operation request instructions, and based on the spin lock method and the parallel access method, executes the atomic cache behavior and the atomic cache memory access behavior according to the atomic operation request instruction, performs atomic on-site centralized parallel processing within the cache, performs in-situ calculation and in-situ storage in the SRAM array, and does not need to transfer data from the memory or L2 cache to the registers in the streaming multiprocessor, thereby reducing data transmission across the storage hierarchy. The spin lock method is further used to manage concurrent access to atomic operations. Combined with the design of the atomic cache, efficient shared variable updates and synchronization are achieved, the overhead of thread switching contexts is avoided, the concurrent execution efficiency is improved, the execution order of atomic operations is relaxed, and at the same time, the semantic conditions of sequential consistency are maintained, thereby obtaining performance gains from removing memory barriers, reducing synchronization overhead and performance losses, and improving algorithm execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 This is a flowchart of a method for implementing atomic operations based on SRAM cache calculations provided by an embodiment of the present application;
[0042] Figure 2 1 is a schematic structural diagram of an atomic operation implementation device based on SRAM cache calculation provided by an embodiment of the present application;
[0043] Figure 3 Schematic diagram of the structure principle of the atomic buffer provided in an embodiment of the present application;
[0044] Figure 4 This is a schematic diagram of the principle of the execution method of the atomic cache provided in the embodiment of the present application;
[0045] Figure 5 This is a schematic flow chart of the steps of executing the atomic cache provided in an embodiment of the present application;
[0046] Figure 6 Schematic diagram of the principle of the execution method of atomic cache memory access provided by the embodiment of the present application;
[0047] Figure 7 This is a flowchart of the steps of executing the atomic cache memory access provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.
[0049] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0050] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.
[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.
[0052] Before explaining the embodiments of the present application in detail, some of the nouns and terms involved in the embodiments of the present application are first explained. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.
[0053] 1) General-Purpose Graphics Processing Unit (GPGPU) refers to technology that uses graphics processing units (GPUs) to perform general-purpose computing tasks. Traditionally, GPUs have been primarily used for graphics rendering and processing. However, as computing demands increase, the parallel computing capabilities of GPUs are being leveraged to accelerate general-purpose computing tasks, such as scientific computing, machine learning, and data analysis. GPGPU's ability to accelerate general-purpose computing tasks is primarily due to the GPU's massive parallel computing capabilities and efficient memory access patterns. However, parallel computing also presents challenges and bottlenecks, including the application of atomic operations and performance bottlenecks.
[0054] 2) Atomic operations are a crucial mechanism for ensuring data consistency and avoiding race conditions. In GPGPUs, atomic operations are often used to protect and synchronize data when multiple threads or cores access shared memory simultaneously. In parallel computing tasks, multiple threads may access the same shared variable simultaneously. Without synchronization mechanisms, data inconsistencies or race conditions can result. In these cases, atomic operations can be used to ensure that operations on shared variables are atomic and uninterruptible, thereby guaranteeing data correctness and consistency.
[0055] There are some deficiencies in the related technology, such as:
[0056] 1) Synchronization Overhead: Atomic operations require synchronization across multiple threads or cores, which introduces additional overhead, particularly when updating and synchronizing shared variables and performing large-scale parallel computations. In large-scale parallel computations, atomic operations require synchronization across multiple threads, increasing data transfer and synchronization overhead. Since processors frequently access shared memory for atomic operations, this can lead to a mismatch in data transfer rates between memory and processors, exacerbating the von Neumann bottleneck.
[0057] 2) Race conditions: Although atomic operations can avoid race conditions, frequent atomic operations can lead to performance degradation because atomic operations require frequent access to multiple storage hierarchies such as registers, shared memory, and caches, which may cause memory contention and exacerbate execution delays caused by the von Neumann bottleneck.
[0058] 3) Storage consistency: Atomic operations are necessary for data consistency, but when processing large-scale data and complex algorithms, frequent atomic operations will fall into the maintenance of a complex storage consistency model, which will significantly reduce the overall performance of the system.
[0059] Therefore, in traditional GPUs, limitations on data reuse and sharing lead to high demands for synchronization. Infrequent synchronization operations are implemented using expensive atomic operations and must be performed at shared ordering points, increasing execution cost and latency, further exacerbating the performance loss caused by the von Neumann bottleneck. In particular, global and cross-thread block synchronization operations in GPGPUs still have high execution costs in practice, further exacerbating the von Neumann bottleneck. As modern GPU applications expand, these applications place higher demands on GPU atomic synchronization and memory consistency.
[0060] In view of this, an embodiment of the present application provides an atomic operation implementation method based on SRAM cache calculation, which explores the feasibility of GPGPU using the interchangeability of atomic operations to circumvent the performance overhead of memory barriers introduced in the ISA specification while maintaining the semantics of the storage consistency model, so that atomic operations can be executed concurrently and centrally in an SRAM-based storage and computing integrated circuit under the condition of ensuring relaxed storage consistency, thereby alleviating the serialization overhead of atomic RMW (read-modified-write) instructions. It not only aims to relax the strict storage consistency model of GPGPU, reduce the execution cost and latency of atomic operations, and provide a more flexible and efficient atomic operation synchronization method, but also is committed to solving the limitations of the von Neumann bottleneck on GPGPU performance and efficiency, so as to meet the urgent needs of modern GPU applications for high-performance, low-cost synchronization, and to improve the performance and scalability of GPGPU systems.
[0061] Reference Figure 1 , Figure 1 A flowchart of a method for implementing atomic operations based on SRAM cache calculations provided by an embodiment of the present invention, referring to Figure 1 , the method comprises the following steps:
[0062] S100, obtaining an atomic operation request instruction;
[0063] S200, based on the spin lock mode and the parallel access mode, according to the atomic operation request instruction, perform the atomic cache behavior and the atomic cache memory access behavior to realize the atomic operation in the cache;
[0064] It should be noted that, in some embodiments, step S200 may include:
[0065] S210, behavior of atomic cache.
[0066] In some specific embodiments, an atomic operation request instruction is obtained and sent to an atomic cache, where the atomic operation request instruction is an atomic read-modify-write operation request instruction; the status of the memory access address of the atomic operation request instruction is judged, and the memory access address status judgment result is output; and an atomic read-modify-write operation is performed based on the memory access address status judgment result.
[0067] Specifically, if the memory address status judgment result is the memory address locked state, the atomic operation request instruction is resent after a preset static time; if the memory address status judgment result is the memory address stateless, the atomic read-modify-write operation is performed according to the atomic operation request instruction, and the memory address stateless state is modified to the memory address locked state, until the atomic read-modify-write operation is completed, and the memory address locked state is modified to the memory address stateless state.
[0068] In this embodiment, if Figure 4 As shown in the figure, to avoid the overhead of context switching while threads are waiting, the atomic cache is implemented using spin locks. This is because the program segments performing atomic operations on data (called critical sections) are not very long. The atomic cache includes lock-holding and lock-releasing procedures before and after the critical section. The SM (streaming multiprocessor) or thread that holds the lock will release it promptly after completing the critical section operation. When an SM or other thread attempts to acquire the spin lock, if the lock is already held, it will repeatedly loop to check if the lock is available until it is acquired. The unique design features of the atomic cache are: first, the atomic cache allows the atomic request of the SM that first applies for the lock to be directly processed in the shared atomic cache through atomic arithmetic and atomic logical operations of the in-memory compute unit, without loading the data into the SM's L2 cache, register file, or compute unit, thus avoiding data transfer across the memory hierarchy. Second, the atomic cache removes the memory barriers required for conventional atomic operations by leveraging the commutativity of typical algorithms, thereby avoiding the performance overhead of introduced memory barriers while maintaining the memory consistency model.
[0069] Commutativity in algorithms is reflected in the fact that some typical GPGPU application algorithms, such as AI training algorithms and graph analysis algorithms, often use commutative, relaxed atomic operations. This means that while access to shared variables must be performed atomically to ensure correctness, the order of atomic operations is not important. This is because the program does not view updated values until all updates have been completed. The atomic operations in such algorithms are commutative and produce the same results regardless of their execution order.
[0070] Based on the above observations, we can also formally show that the order in which competing commutative atomic operations are executed does not affect the final result of the execution and cannot be used to infer the order of other operations in the execution. Furthermore, after all competing, conflicting commutative atomic operations, the load of the final value is always separated by pairing synchronization; therefore, the load does not depend on the ordering of the commutative atomic operations (relative to other loose atomic operations) to obtain the correct value.
[0071] Therefore, by distinguishing and leveraging program commutativity, we can relax the execution order of atomic operations while still meeting semantic requirements centered on sequential consistency (supporting a correct storage consistency model), thereby achieving the performance gains brought about by removing memory barriers. This means that under certain conditions, shared variable updates and synchronization can not only be atomically and centrally processed in parallel within the cache, but also achieve more efficient algorithm execution, reducing unnecessary synchronization overhead and performance loss.
[0072] In summary, refer to Figure 5 , the steps of how atomic cache is executed are as follows:
[0073] 1) Identify atomic operation instructions that comply with the RISC-V instruction set specification;
[0074] 2) Atomic read-modify-write requests are sent to the atomic cache;
[0075] 3) Similar to the spin lock method, the atomic request checks whether the memory address in the atomic cache holds a lock. If the memory address already holds a lock, it proves that the memory address is performing another atomic operation, and the atomic read-modify-write request is re-broadcasted later;
[0076] 4) If the memory address is not locked, lock it to prevent other operations from interrupting the atomic operation process;
[0077] 5) Execute the corresponding in-memory atomic operation in the cache according to the atomic operation type;
[0078] 6) After the atomic operation is completed, the lock for the atomic cache memory address is released.
[0079] S220: Atomic cache memory access behavior.
[0080] In some specific embodiments, an atomic operation request instruction is obtained and sent to an atomic cache, where the atomic operation request instruction is an atomic read-modify-write operation request instruction; a cache tag judgment is performed based on the atomic operation request instruction, and a cache judgment result is output; and an atomic read-modify-write operation is performed based on the cache judgment result.
[0081] Among them, if the cache judgment result indicates a cache miss, a memory access request is initiated to the shared global memory and a memory access request reply is obtained; the memory access request reply is sent to the atomic cache to establish a cache tag and perform an atomic read-modify-write operation; if the cache judgment result indicates a cache hit, an atomic read-modify-write operation is directly performed.
[0082] In addition, it should be noted that if the first atomic operation request instruction and the second atomic operation request instruction have the same cache tag, and when the cache judgment result indicates a cache hit, the first atomic operation request instruction and the second atomic operation request instruction are executed in parallel.
[0083] In this embodiment, if Figure 6 As shown in Figure 1, the atomic cache is located in the shared memory hierarchy between the SM and the global memory. It has the storage and read and write functions of a normal cache (SRAM mode) and the function of atomic operations in the cache (AMO mode). Taking the memory access behavior of SMs as an example, the SM issues an atomic request (Atomic RMW Request) to perform an atomic operation of adding 1 to the memory unit at address addr0. Since it is the first access to this address, the request will encounter a cache miss until the data in the global memory is returned to the atomic cache, and then the in-cache atomic addition (such as Figure 6 When another atomic request needs to perform an atomic operation on the value of the same memory unit, the request will hit the atomic cache and be directly calculated in place in the atomic storage unit, thereby fully utilizing the spatiotemporal locality of the atomic cache (e.g. Figure 6 In addition, if a cache miss request to access addr1 is waiting for data, and another SM issues an atomic request for the same address, these two requests can be combined in the atomic cache to improve the calculation and memory access efficiency (e.g. Figure 6 Serial number ⑧~ ).
[0084] In summary, refer to Figure 7 , the steps of atomic cache memory access are as follows:
[0085] 1) Atomic read-modify-write request is sent to the atomic cache;
[0086] 2) It is necessary to check whether the address requested by the atomic read-modify-write request hits in the atomic cache. If it hits, the corresponding in-memory atomic operation can be directly executed in the cache according to the atomic operation type to perform a read-modify-write operation on the memory unit;
[0087] 3) If the request misses, a memory access request is made to the global memory;
[0088] 4) Wait for the data of the memory access request to be returned. If the data has been returned, execute the corresponding in-memory atomic operation in the cache according to the atomic operation type, and perform a read-modify-write operation on the memory unit. Otherwise, wait.
[0089] See also Figure 2 The embodiment of the present application further provides an atomic operation implementation device based on SRAM cache calculation, which can implement the above-mentioned atomic operation implementation method based on SRAM cache calculation. The device includes a thread block scheduler, multiple streaming multiprocessors, an L2 cache, and an atomic cache. The output ends of the thread block scheduler are connected to the input ends of the multiple streaming multiprocessors, the first input end of the L2 cache, and the first input end of the atomic cache. The output ends of the multiple streaming multiprocessors are connected to the second input end of the L2 cache and the second input end of the atomic cache, wherein:
[0090] The thread block scheduler is used to control the scheduling and allocation of thread execution order and generate thread execution instructions;
[0091] Several streaming multiprocessors are used to execute thread execution instructions and coordinate data synchronization and sharing among threads;
[0092] The L2 cache is used to cache data for executing instructions of the execution thread;
[0093] Atomic buffers are used to manage atomic operations for updating and synchronizing shared variables.
[0094] Furthermore, it should be noted that the streaming multiprocessor includes an instruction cache and a data cache, wherein the instruction cache is used to accelerate access operations of thread execution instructions; and the data cache is used to accelerate access operations of data between coordinated threads.
[0095] In this embodiment, the streaming multiprocessor further includes a shared global memory, and the shared global memory is used for data interaction between the streaming multiprocessor and the L2 cache and the atomic cache.
[0096] Specifically, the thread block scheduler in GPGPU is responsible for managing and assigning the execution order of threads to improve the efficiency of parallel computing. The core of GPGPU is multiple streaming multiprocessors (SMs), which are responsible for executing instructions in parallel and coordinating data sharing and synchronization between threads. Each SM has an instruction cache and a data cache, as well as an optional shared memory. The instruction cache accelerates instruction access and reduces global memory access latency; the data cache accelerates data access, reducing memory bandwidth usage and latency; and the shared memory provides a fast, low-latency data exchange mechanism for efficient communication and data sharing among threads within the same thread block. Multiple SMs form a cluster, which is optionally connected to a shared L2 cache and atomic cache. This implements a standard five-level in-order RISC-V pipeline. The atomic cache is a key component of the execution stage of the pipeline, managing atomic operations for updating and synchronizing shared variables.
[0097] In addition, refer to Figure 3 The atomic buffer includes an on-site storage type 8T SRAM array, a switch control circuit, a near-storage calculation logic, a data drive control circuit and an SRAM control circuit. The output end of the switch control circuit is connected to the first input end of the on-site storage type 8T SRAM array, the output end of the near-storage calculation logic is connected to the input end of the data drive control circuit, the output end of the data drive control circuit is connected to the second input end of the on-site storage type 8TSRAM array, and the output end of the SRAM control circuit is connected to the third input end of the on-site storage type 8TSRAM array. The on-site storage type 8T SRAM array is used to store data generated by atomic operations in memory and control the calculation of atomic operations in memory; the switch control circuit is used to control the on-site storage type 8TSRAM array to switch between SRAM mode and atomic operation mode; the near-storage calculation logic is used to control the carry calculation of atomic operations in memory; the data drive control circuit is used to control a number of atomic operations in memory; and the SRAM control circuit is used to control data access of the on-site storage type 8T SRAM array.
[0098] In this embodiment, the main circuit of the atomic cache is composed of an on-site storage type 8T SRAM array, a switch control circuit, a near-memory calculation logic, a data drive control circuit, and an SRAM control circuit. Among them, the on-site storage type 8T SRAM array is responsible for completing data storage and in-memory atomic operations; the switch control circuit is used to control Ctrl_L and Ctrl_R in the SRAM array to achieve switching between SRAM mode and AMO mode; the near-memory calculation logic is used to generate the carry signal and comparison signal required to perform atomic arithmetic operations (amoadd, amomax, and amomin); the data drive control circuit is used to control IN1 and IN2 in the storage array to support multiple atomic operations; the SRAM control circuit is responsible for managing the regular read and write operations of the standard SRAM. Depending on the opcode, the atomic cache can perform different atomic logical operations and atomic arithmetic operations, as shown in Table 1.
[0099] Table 1 Schematic diagram of atomic operation logic
[0100]
[0101] In AMO mode, AND, OR, XOR, and SWAP operations all complete in one cycle. The SWAP operation can be simply considered as writing operand 2 to the address. Therefore, the write operation also requires one cycle. ADD, MAX, and MIN operations, on the other hand, require three cycles. This design aims to efficiently support diverse operations, ensuring circuit flexibility and performance.
[0102] In summary, the improvements of the embodiments of the present invention are specifically as follows:
[0103] 1) In-memory computing architecture design: This architecture leverages the advantages of in-memory computing in the SRAM cache to execute updates and synchronization operations of global shared variables in the cache. This avoids frequent data movement in the storage hierarchy, including registers, shared memory, and cache, and improves data access efficiency.
[0104] 2) Atomic Operation Optimization: By distinguishing and leveraging commutativity in programs, we relax the execution order of atomic operations while maintaining sequential consistency semantics, thereby achieving performance gains from removing memory barriers. This optimization reduces synchronization overhead and performance loss, improving algorithm execution efficiency.
[0105] 3) Parallel processing of atomic operations within memory: Atomic, centralized parallel processing is performed in-place within the cache, with in-situ computation and storage performed in the SRAM array. This eliminates the need to transfer data from memory or L2 cache to registers in the SM, reducing data transfer across the memory hierarchy and further improving performance.
[0106] 4) Combination of spin locks and atomic caches: Spin locks are used to manage concurrent access to atomic operations. Combined with the design of atomic caches, efficient shared variable updates and synchronization are achieved, avoiding the overhead of thread context switching and improving concurrent execution efficiency.
[0107] 5) Utilization of spatiotemporal locality: Atomic caches fully utilize the spatial and temporal locality of atomic caches to achieve efficient in-memory computing and optimize the performance of shared variable updates and synchronization.
[0108] This embodiment of the present invention leverages the inherent computational properties of SRAM cache to perform updates and synchronization of global shared variables within a GPGPU, reducing the need for frequent data movement within the storage hierarchy and improving data access efficiency. This design enables in-memory computation, providing a foundation for subsequent optimizations. Furthermore, by distinguishing and leveraging program commutativity, the execution order of atomic operations is relaxed, achieving performance gains from removing memory barriers while reducing synchronization overhead and performance losses. While maintaining the semantic conditions of sequential consistency, the algorithm execution efficiency is improved, and further atomic local and centralized parallel processing is performed in the cache. There is no need to transfer data from the memory or L2 cache to the registers in the SM, which reduces the data transmission across the storage hierarchy and realizes the local centralized processing of data in the SRAM storage array, reduces the processing delay, and improves the performance. The spin lock is used to manage the concurrent access of atomic operations. Combined with the design of the atomic cache, the concurrent access of atomic operations is efficiently managed, the overhead of thread switching context is avoided, and the concurrent execution efficiency and the parallel processing capability of the system are improved. Finally, the spatial locality and temporal locality of the atomic cache are fully utilized to achieve efficient in-memory computing, optimize the performance of shared variable updates and synchronization, reduce the delay of data access, and improve the overall performance of the system by effectively utilizing the resources in the cache.
[0109] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0110] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.
Claims
1. A method for implementing atomic operations based on SRAM cache calculation, characterized in that: The method comprises the following steps: Get atomic operation request instruction; Based on the spin lock mode and the parallel access mode, according to the atomic operation request instruction, the atomic cache behavior and the atomic cache memory access behavior are executed to realize the atomic operation in the cache.
2. The method according to claim 1, characterized in that The atomic cache behavior includes: Acquire the atomic operation request instruction and send it to the atomic buffer, wherein the atomic operation request instruction is an atomic read-modify-write operation request instruction; Determine the state of the memory access address of the atomic operation request instruction, and output the memory access address state determination result; An atomic read-modify-write operation is performed according to the memory access address state judgment result.
3. The method according to claim 2, characterized in that The performing of the atomic read-modify-write operation according to the memory access address state judgment result includes: If the memory access address state judgment result is a memory access address locked state, resending the atomic operation request instruction after a preset static time; If the memory access address status judgment result is that the memory access address is stateless, the atomic read-modify-write operation is performed according to the atomic operation request instruction, and the memory access address stateless is modified to the memory access address locked state, until the atomic read-modify-write operation is completed, and the memory access address locked state is modified to the memory access address stateless.
4. The method according to claim 1, characterized in that: The atomic cache memory access behavior includes: Acquire the atomic operation request instruction and send it to the atomic buffer, wherein the atomic operation request instruction is an atomic read-modify-write operation request instruction; Perform cache tag judgment according to the atomic operation request instruction, and output the cache judgment result; The atomic read-modify-write operation is performed according to the cache determination result.
5. The method according to claim 4, characterized in that Performing the atomic read-modify-write operation according to the cache determination result includes: If the cache determination result indicates a cache miss, a memory access request is initiated to the shared global memory, and a memory access request reply is obtained; Sending the memory access request reply to the atomic cache to establish a cache tag and perform the atomic read-modify-write operation; If the cache determination result indicates a cache hit, the atomic read-modify-write operation is performed directly.
6. The method according to claim 5, characterized in that Also includes: If the first atomic operation request instruction and the second atomic operation request instruction have the same cache tag, and when the cache determination result indicates a cache hit, the first atomic operation request instruction and the second atomic operation request instruction are executed in parallel.
7. An atomic operation implementation device based on SRAM cache calculation, characterized in that: The device comprises a thread block scheduler, a plurality of streaming multiprocessors, an L2 cache and an atomic cache, wherein the output end of the thread block scheduler is connected to the input ends of the plurality of streaming multiprocessors, the first input end of the L2 cache and the first input end of the atomic cache, and the output ends of the plurality of streaming multiprocessors are connected to the second input end of the L2 cache and the second input end of the atomic cache, wherein: The thread block scheduler is used to control the scheduling and allocation of thread execution order and generate thread execution instructions; The plurality of streaming multiprocessors are used to execute the thread execution instructions and coordinate data synchronization and sharing among threads; The L2 cache is used to cache data for executing the thread execution instruction; The atomic buffer is used to manage atomic operations for updating and synchronizing shared variables.
8. The device according to claim 7, characterized in that The streaming multiprocessor includes an instruction cache and a data cache, wherein: The instruction cache is used to accelerate the access operation of the thread execution instruction; The data buffer is used to accelerate the data access operation between the coordination threads.
9. The device according to claim 8, characterized in that The streaming multiprocessor further includes a shared global memory, and the shared global memory is used for data interaction between the streaming multiprocessor and the L2 cache and the atomic cache.
10. The device according to claim 7, characterized in that The atomic buffer comprises an on-site storage type 8T SRAM array, a switch control circuit, a near storage calculation logic, a data drive control circuit and an SRAM control circuit, wherein the output end of the switch control circuit is connected to the first input end of the on-site storage type 8T SRAM array, the output end of the near storage calculation logic is connected to the input end of the data drive control circuit, the output end of the data drive control circuit is connected to the second input end of the on-site storage type 8T SRAM array, and the output end of the SRAM control circuit is connected to the third input end of the on-site storage type 8TSRAM array, wherein: The field storage type 8T SRAM array is used to store data generated by in-memory atomic operations and control the calculation of the in-memory atomic operations; The switch control circuit is used to control the field storage type 8T SRAM array to switch between the SRAM mode and the atomic operation mode; The near memory calculation logic is used to control the carry calculation of the in-memory atomic operation; The data drive control circuit is used to control a number of the atomic operations in the memory; The SRAM control circuit is used to control data access of the field storage type 8T SRAM array.
Citation Information
Cited By
Method for executing atomic addition operation, artificial intelligence chip, computing device, medium and program product
CN121209825A
Cache operation method, cache, computing device and system
CN121958146A
Cache operation method, cache, computing device and system
CN121958146B
