Method, apparatus, and computer program product for cross-chip atomic operation

Abstract

The invention discloses a method, an apparatus, and a computer program product for cross-chip atomic operation. The method comprises the steps that on a source computing chip, an atomic operation instruction initiated by a thread is intercepted, and a target memory address corresponding to the atomic operation instruction is determined; in response to determining that the target memory address belongs to the remote computing chip, forwarding the atomic operation instruction to an atomic operation proxy cache, the atomic operation proxy cache being arranged in an intermediate layer connecting the source computing chip and the remote computing chip; querying by the atomic operation proxy cache according to the target memory address; in response to the query hit, determining whether the source computing chip has an exclusive modification permission for the target memory address; in response to determining that the source computing chip has the exclusive modification permission, atomic operations are performed locally by the source computing chip. According to the method, high-frequency remote atomic operation is dynamically converted into local execution, so that the technical bottlenecks of high delay and low throughput of cross-chip atomic operation in a multi-chip GPU system are solved.

Description

Technical Field

[0001] This disclosure generally relates to the field of computers, and more specifically to methods, apparatus and computer program products for atomic memory operations (AMO) across chips. Background Technology

[0002] With the increasing demand for computing power from applications such as artificial intelligence, chiplet technology has become the mainstream approach for building high-performance GPUs by integrating multiple chips or dies. These multi-chip systems logically share a unified memory address space, but physically, data is distributed across the memory of different chips. Cross-chip atomic operations occur when a thread on one chip needs to perform atomic operations (such as atomic addition) on data on another chip.

[0003] In multi-chip GPU architectures, cross-chip atomic operations (AMOs) are typically performed on remote chips or in memory, and global atomicity and consistency must be guaranteed. Therefore, existing solutions suffer from high latency and high hardware overhead, severely limiting the overall performance of multi-chip GPU systems. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this application proposes an improved method for cross-chip atomic operations, as well as an associated apparatus and computer program product.

[0005] According to a first aspect of this disclosure, a method for cross-chip atomic operations is provided. The method includes: on a source computing chip, intercepting an atomic operation instruction initiated by a thread and determining a target memory address corresponding to the atomic operation instruction; in response to determining that the target memory address belongs to a remote computing chip, forwarding the atomic operation instruction to an atomic operation proxy cache associated with the source computing chip, wherein the atomic operation proxy cache is configured in an intermediary layer connecting the source computing chip and the remote computing chip; querying the atomic operation proxy cache based on the target memory address; in response to a successful query, determining whether the source computing chip has exclusive modification rights to the target memory address; and in response to determining that the source computing chip has exclusive modification rights, performing the atomic operation locally on the source computing chip.

[0006] According to a second aspect of this disclosure, an apparatus for cross-chip atomic operations is provided. The apparatus includes: a plurality of computing chips; and an intermediary layer connecting the plurality of computing chips. The intermediary layer includes an atomic operation proxy cache associated with each of the plurality of computing chips for performing the methods described in this application.

[0007] In other aspects of this disclosure, a computer program product including instructions is provided. These instructions, when executed by one or more processors, cause the processors to perform the methods described in this application. Attached Figure Description

[0008] The operation and function of these and other features disclosed herein, as well as related structural elements and combinations thereof, will become more apparent upon consideration of the following description and appended claims with reference to the accompanying drawings (all of which form a part of this specification, wherein like reference numerals denote corresponding portions in the drawings). However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and not for limiting purposes. In the drawings: Figure 1 A schematic diagram of a system architecture for cross-chip atomic operations according to an embodiment of the present disclosure is shown.

[0009] Figure 2 A schematic diagram of the structure of a single entry for an atomic operation proxy cache (AMO Proxy Cache, APC) according to an embodiment of the present disclosure is shown.

[0010] Figure 3 A flowchart of a method for cross-chip atomic operations according to an embodiment of the present disclosure is shown.

[0011] Figure 4 An embodiment according to this disclosure is shown. Figure 1 The system shown is a schematic diagram of an example computing device in which it can be used. Detailed Implementation

[0012] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and not limiting.

[0013] Currently, in fields such as computer system software, high-performance computing, and hardware design, atomic operations (including atomic add and atomic compare-and-swap) are used to ensure the correctness of multithreaded parallel programs. In traditional monolithic GPUs, atomic operations are handled directly by a shared Last Cache (LLC) controller or memory controller, resulting in extremely low latency. However, in multi-chip chip systems, when a thread on one computing chip (source chip) needs to perform atomic operations on data physically located in the memory of another computing chip (remote chip or home chip), cross-chip atomic operations (Cross-Die AMO) occur. To address this, the following techniques can be employed in multi-chip chip systems to improve cross-chip atomic operations.

[0014] Some technologies employ software emulation protocols. These protocols, through the operating system or runtime software libraries, translate hardware atomic instructions into software algorithms based on locks or message passing. While this technique is versatile, it introduces significant software overhead and multiple inter-chip communications, resulting in latency for cross-chip atomic operations that is several orders of magnitude higher than for on-chip operations, thus becoming a system performance bottleneck.

[0015] Some technologies employ hardware coherence protocols based on a global directory. These protocols maintain a global directory within the system to track cache line states. When cross-chip atomic operations occur, a complex protocol involving multiple rounds of inter-chip communication (requests, forwarding, responses, etc.) is required to complete the operation on the data-owning chip. Although implemented in hardware, this technology suffers from protocol complexity, significantly higher communication latency than on-chip operations, and the global directory structure itself consumes substantial amounts of valuable on-chip storage resources, potentially leading to scalability issues.

[0016] Therefore, these technologies generally suffer from high latency, large hardware overhead, and complex protocols when handling cross-chip atomic operations under the chip architecture, making it difficult to meet the requirements of high-performance computing applications for low-latency, high-throughput synchronous operations.

[0017] To address the aforementioned issues, this application proposes a cross-chip atomic operation scheme based on an Atomic Operation Proxy Cache (APC). In this scheme, a dedicated APC can be configured for each chip in the intermediary layer connecting the various computing chips. This APC is not a simple forwarder, but rather serves as a global serialization and scheduling center for cross-chip atomic operations. When a source chip initiates a cross-chip atomic operation, the request is first routed to the local APC. The APC queries its internal state table based on the target address. If the query matches, the operation is executed directly in the source chip's local cache. Otherwise, the APC buffers the request and can intelligently merge it with other atomic operations targeting the same cache line. When predetermined conditions are met, the merged requests are sent in batches to the remote home chip for execution.

[0018] This scheme has some or all of the following characteristics.

[0019] 1. Reduce latency: By using the "state hit-local execution" mechanism, high-frequency remote atomic operations are transformed into local operations, eliminating cross-chip network round-trip latency.

[0020] 2. Reduce overhead and increase throughput: By using APC's merge buffer mechanism, a large number of fine-grained requests are merged into a small number of batch requests, which greatly reduces the number of network communications and the atomic operation execution load of remote chips, thereby improving the overall synchronous throughput of the system.

[0021] 3. Simplified Protocol and Hardware Design: As a dedicated serialization point, APC only manages and coordinates cross-chip atomic operations, without participating in the consistency protocol of ordinary cache load / store operations. This simplifies the overall hardware design and avoids the complexity and resource consumption of a global directory. For all cross-chip atomic operations on the same cache line, APC ensures that they are executed strictly in order, thereby guaranteeing operational consistency while achieving high performance.

[0022] Figure 1 A schematic diagram of a system architecture for cross-chip atomic operations according to embodiments of the present disclosure is shown. Figure 1 As shown, system 100 may include multiple computing chips, such as a first computing chip (die A) 110 and a second computing chip (die B) 120. These computing chips are physically and electrically interconnected through a high-bandwidth, low-latency interposer 140. Each computing chip includes a computing core (e.g., a streaming multiprocessor array (SM) 111 in die A 110 and a streaming multiprocessor array SM 121 in die B 120, wherein SM 111 and 121 include L1 caches), cache hierarchies (e.g., respective last-level caches L2 caches 112 and 122), and a memory controller (not shown).

[0023] In embodiments of this application, an atomic operation proxy cache (APC) 150 is disposed in an intermediary layer 140. Specifically, system 100 may include multiple APCs, each logically and exclusively associated with a computing chip. Figure 1 As shown, APC_A 151 is associated with die A 110, and APC_B 152 is associated with die B 120. APCs 151 and 152 can communicate directly with their associated computing chips via a first dedicated channel (as indicated by the arrow between the APC and the die) to receive atomic operation requests and return results. Furthermore, APCs 151 and 152 can be interconnected via a second dedicated channel (as indicated by the arrow between the APCs), forming an APC interconnect network covering the interposer layer 140 for transmitting cross-chip atomic operation request and response packets. A wider inter-chip interconnect network (not shown separately in the figure) can also be integrated within the interposer layer 140 for general data transmission. The dedicated channel between APCs can be viewed as a high-priority virtual channel or physically isolated channel implemented on this network.

[0024] For example, when a thread on die A 110 initiates an atomic operation on a memory address of die B 120, the request is first forwarded to its associated APC_A 151. APC_A 151 then queries the internal status of the target address. If the query is successful and the status bit of the corresponding entry indicates that die A 110 has exclusive modification rights to the data (e.g., "Exclusive" or "Modified"), APC_A 151 will authorize the operation to be executed directly in the local L2 cache 112 of die A 110, thereby completing the operation with extremely low latency, similar to an on-chip operation, avoiding cross-chip communication. Conversely, if the query is unsuccessful or the status bit indicates no exclusive rights (e.g., "Shared" or "Invalid"), APC_A 151 will buffer the request for subsequent merging and scheduling. Finally, the processed request is sent to the target chip die B 120 for execution via the APC network.

[0025] It is important to note that Figure 1 The figures are merely simplified examples to clearly illustrate the principles of the invention. Those skilled in the art will understand that the embodiments of this application are not limited to the two computing chips shown in the figures. Depending on the actual application requirements, system 100 may include three, four, or any number of computing chips interconnected through an intermediary layer. Each computing chip is associated with a dedicated APC, and all APCs together constitute a complete cross-chip atomic operation coordination network.

[0026] Figure 2 A schematic diagram of the structure of a single entry 200 for an atomic operation proxy cache (APC) according to an embodiment of the present disclosure is shown. The APC may include a dedicated hardware table for managing and coordinating cross-chip atomic operations. In embodiments of this application, the APC is managed at a cache line level, meaning each entry 200 in the APC corresponds to a cache line in system memory, responsible for tracking and scheduling all cross-chip atomic operations targeting any address within that cache line.

[0027] like Figure 2 As shown, a typical APC entry 200 may include at least one or more of the following fields: cache line address 201, status bit 202, compute chip identifier 203, atomic operation accumulation value 204, pending flag 205, and merge counter 206.

[0028] The cache line address 201 serves as the index key for the entry, uniquely identifying the physical memory block managed by that entry. The status bit 202 defines the permission status of the cache line within the context of cross-chip atomic operations, directly determining the processing path for subsequent requests. In one embodiment, status bit 202 may include, but is not limited to, the following states.

[0029] Invalid (I): This indicates that the APC entry does not currently hold any valid status information, which usually corresponds to the entry being just assigned or the previous operation being completely completed.

[0030] Shared (S): This indicates that the APC is aware of this cache line, and multiple chips may have read access to it, but no chip (including the source chip associated with this APC) has exclusive modification rights. The latest version of the data still resides on its physical home chip. In this state, the APC primarily acts as a request merger, buffering and merging multiple atomic operations for this line.

[0031] Exclusive (O) or Modified (M): Both states indicate that the source compute chip associated with this APC has obtained exclusive modification rights to the cache line. This means that the data copy of the cache line has not only been migrated to the source chip's local cache, but the system has also ensured through a consensus protocol that no other chip will hold a modifiable copy after this migration and before the state changes. The Modified (M) state further indicates that the local copy has been modified and is updated compared to the data in the home chip's memory. When the state bits are both binary, it means that atomic operations on the cache line can be completed locally on the source chip with extremely low latency, forming a "fast path" that avoids cross-chip communication.

[0032] Furthermore, the remote computing chip identifier 203 can record the identity of the physical home chip (Home Die) of this cache line data to indicate the target when remote execution is required. The atomic operation accumulator 204 is used to accumulate the operands of multiple assimilable atomic operations (such as atomicAdd) targeting the same cache line when the state is "Shared" (S). The pending flag 205 can indicate whether a request has been made to the remote chip for this entry but a completion response has not yet been received, used to manage the request pipeline. The merge counter 206 can record the number of atomic operation instructions accumulated in the atomic operation accumulator 204, ensuring the accuracy and traceability of merge operations. Some or all of these fields together constitute the data foundation for APC to make intelligent decisions (such as determining whether local execution is possible) and optimize scheduling (such as requesting merge).

[0033] It should be pointed out that, Figure 2 The illustrated entry structure and its included fields are merely an example to clearly illustrate the working principle of APC. The scope of this application is not limited to this specific structure. For example, in actual implementations, depending on different design optimization goals, APC entries may contain more, fewer, or different fields than illustrated, or some fields may be merged, split, or encoded using different methods. As long as it achieves the functions of state tracking, path decision-making, and request scheduling for cross-chip atomic operations, it should fall within the scope of this invention.

[0034] Figure 3 A flowchart of a method 300 for cross-chip atomic operations according to an embodiment of the present disclosure is shown. Method 300 can be, for example... Figure 1 The system 100 shown executes, and mainly includes the following operations.

[0035] When operating the S310, in the source computing chip (e.g., Figure 1 On the die A110, when a thread within the execution core (such as SM array 111) initiates an atomic operation instruction (e.g., atomicAdd(addr, delta), indicating an atomic addition of delta to memory address addr), the instruction is first intercepted by the chip's hardware memory subsystem (typically integrated in the load / store unit or cache controller). This subsystem then parses the instruction to determine the target memory address addr corresponding to the atomic operation instruction. This process is transparent and unaware of the upper-level software threads.

[0036] In operation S320, the memory subsystem can determine the physical ownership of an address based on a determined addr (e.g., by querying an address mapping table maintained internally by the chip or by parsing the high-order bits of the addr). This mapping defines the physical memory chip (i.e., its home die) corresponding to each address range in the system's unified address space. Operation S320 also includes responding to determining that the target memory address belongs to a remote computing chip (e.g., Figure 1 In the B120 die (of which the memory subsystem will follow the specific processing path of this application), it will not attempt to execute locally or initiate complex inter-chip consistency protocols. Specifically, the memory subsystem can forward atomic operation instructions to an atomic operation proxy cache associated with the source computing chip. Figure 1 As shown, the atomic operation proxy cache (APC_A 151) is set in the intermediary layer 140 that connects multiple computing chips and is connected to die A 110 via a dedicated channel. The forwarded information typically includes: the target memory address addr, the atomic operation type (such as ADD), the operand (such as delta), and the context identifier of the source thread, etc.

[0037] In operation S330, the atomic operation proxy cache (e.g., APC_A151) performs a query operation after receiving a forwarded atomic operation request. Specifically, APC can perform a query based on the target memory address. In some embodiments, this query uses the cache line address to which the target memory address belongs as an index to query the entry table of the atomic operation proxy cache. In some embodiments, the entry table of APC (whose entry structure is as follows) Figure 2(As shown) can be a hardware table organized by cache line address. The AP can extract the cache line address corresponding to addr (usually obtained by masking the low-order bits of the address) and use this address to search its entry table to check if there is a matching entry.

[0038] In operation S340, in response to a query hit (i.e., an entry corresponding to the cache line address is found in the APC's entry table), the source computing chip (e.g., die A110) locally performs the atomic operation. In some embodiments, before the source computing chip performs the atomic operation, it can be further determined whether the conditions for local execution are met. For example, the status of the entry corresponding to the target memory address in the atomic operation proxy cache can indicate that the source computing chip has exclusive modification rights to the target memory address, and the source computing chip performs the atomic operation locally.

[0039] Specifically, atomic operation proxy caches (such as APC_A 151) can check the status bits of hit entries in addition to checking whether a hit occurred. Figure 2 (202 in the text above). Figure 2 The status bits include Exclusive (O) or Modified (M) states. When the status bit is binary, its physical meaning is: 1) the source computing chip (die A) has obtained exclusive access to the cache line through some previous mechanism (such as ownership transfer); 2) the data corresponding to the target memory address is cached in the source computing chip's local cache (e.g., in die A's L2 cache 112). Therefore, APC_A 151 does not need to communicate with any remote chip and can directly send a local execution grant to the cache controller of die A 110. The cache controller of die A 110 then performs an atomic "read-modify-write" operation on the data copy at address addr in its local cache. This path avoids inter-chip network latency and achieves performance similar to intra-chip atomic operations.

[0040] In some embodiments, method 300 may further include additional operations. For example, in response to a query miss or a query hit but the status of the entry indicates that the source computing chip does not have exclusive modification rights (e.g., the status bit is "Invalid (I)" or "Shared (S)"), APC_A 151 will not trigger local execution. At this time, the system enters a "buffered merge path" to optimize throughput and specifically performs the following operations.

[0041] First, in operation S350, atomic operation instructions are buffered by the atomic operation proxy cache. The atomic operation proxy cache (such as APC_A 151) can create a new entry in the entry table for this request (if a miss) or utilize an existing entry (if a hit but in state S) and mark it as pending.

[0042] Next, in operation S360, multiple atomic operation instructions targeting the same target memory address in the atomic operation proxy cache are merged. Specifically, the atomic operation proxy cache can check whether the new request and other cached requests in the entry target the same cache line and are combinable atomic operations (such as multiple atomicAdd). If so, the atomic operation proxy cache can accumulate the operands of each atomic operation to the atomic operation accumulation value of the corresponding entry in the atomic operation proxy cache. Figure 2 In the 204 field, update the merge counter ( Figure 2 (206 in the example). In this way, multiple independent fine-grained requests are merged into a single logical aggregate request in APC.

[0043] In some embodiments, the atomic operation proxy cache can wait for preset triggering conditions (such as timer timeout, accumulated value reaching a threshold, encountering a memory barrier, etc.), and then send the merged single atomic operation request to the remote home chip (die B) for execution, thereby reducing multiple expensive cross-chip communications to one, significantly improving the overall system throughput.

[0044] In some embodiments, the preset triggering conditions include at least one of the following: a timer timeout since the first atomic operation instruction was buffered; the merged operation value reaches a preset threshold; a memory barrier instruction or a synchronization instruction is encountered; or an access request for the target memory address is received from another computing chip.

[0045] By combining the above operations, the embodiments of the present invention intelligently and dynamically select between a "low-latency fast path" (local execution) and a "high-throughput optimized path" (merging buffer), thereby simultaneously optimizing the latency and bandwidth of cross-chip atomic operations and effectively solving the technical bottleneck in multi-chip GPU systems.

[0046] Figure 4 An embodiment according to this disclosure is shown. Figure 1 The system shown is a schematic diagram of an example computing device in which it can be used. (See diagram for example.) Figure 4 As shown, computing device 400 may include bus 402 or other communication mechanism for transmitting information, and one or more hardware processors 404 coupled to bus 402 for processing information. The one or more hardware processors 404 may include, for example, one or more general-purpose microprocessors.

[0047] like Figure 4As shown, in some embodiments, computing device 400 may further include main memory 406 coupled to bus 402. Main memory 406 is used to store information and instructions executed by one or more processors 404, such as random access memory (RAM), cache, and / or other dynamic storage devices. Main memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions executed by one or more processors 404. When these instructions are stored in storage media accessible to one or more processors 404, they can cause computing device 400 to become a dedicated machine customized to perform the operations specified in the instructions. Storage device 408 may include non-volatile and / or volatile storage media. Non-volatile storage media may include, for example, optical discs or magnetic disks. Volatile storage media may include dynamic memory. Common forms of storage media may include, for example, floppy disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, DRAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, or networking versions thereof.

[0048] like Figure 4 As shown, in some embodiments, computing device 400 may further include one or more communication interfaces or network interfaces 410 coupled to bus 402. Network interface 410 may provide bidirectional data communication coupling to one or more network links connected to one or more networks. As another example, network interface 410 may be a local area network (LAN) card to provide data communication connectivity to a LAN-compatible (or WAN component communicating with a WAN) network. Wireless links may also be implemented.

[0049] The execution of certain operations can be distributed across processors rather than residing within a single machine, but rather deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0050] Each of the processes, methods, and algorithms described in the preceding sections may be embodied in code modules executed by one or more computer systems or computer processors including computer hardware, and may be fully or partially automated by these code modules. The processes and algorithms may be implemented, partially or fully, in dedicated circuit systems.

[0051] When the functions disclosed herein are implemented as software functional units and sold or used as stand-alone products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product may be stored in a storage medium and includes several instructions that cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this application. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, other media operable to store program code, or any combination thereof.

[0052] Specific embodiments further provide a system including a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps in any method of the embodiments disclosed above. Specific embodiments further provide a non-transitory computer-readable storage medium storing instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to steps in any method of the embodiments disclosed above.

[0053] The embodiments disclosed herein can be implemented via a cloud platform, server, or server cluster (collectively referred to below as the "Service System") that interacts with a client. The client can be a terminal device or a client registered by a user at the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.

[0054] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and their associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. Example blocks or states may be executed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently than described. For example, components may be added to, removed from, or rearranged compared to the disclosed example embodiments.

[0055] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. The algorithm may be included in program code or instructions stored in memory (e.g., the aforementioned non-transitory computer-readable storage medium). This algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly refer to the computer as performing the function but may learn from training data to generate a predictive model of the function.

[0056] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, these processors can constitute an engine of processor implementation that operates to perform one or more of the operations or functions described herein.

[0057] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are instances of hardware. For example, at least some operations of the methods may be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors may also be operable to support the execution of relevant operations in a “cloud computing” environment or as the execution of relevant operations in a “Software as a Service” (SaaS) context. For example, at least some operations may be performed by a group of computers (as an example of a machine containing processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0058] The execution of certain operations can be distributed across processors rather than residing within a single machine, and can be deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0059] Throughout this specification, multiple instances may be implemented as components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of these individual operations may be performed simultaneously, and not necessarily in the order illustrated. Structures and functions presented as separate components in the example configuration may be implemented as composite structures or components. Similarly, structures and functions presented as single components may be implemented as single components. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0060] As used herein, "or" is inclusive rather than exclusive unless explicitly indicated by the context. Therefore, in this document, "A, B, or C" means "A, B, A and B, A and C, B and C, or A, B, and C" unless explicitly indicated by the context. Furthermore, "and" is combined and separate unless explicitly indicated by the context. Therefore, in this document, "A and B" means "A and B, combined or separate" unless explicitly indicated by the context. Additionally, multiple instances of resources, operations, or structures described herein may be provided as a single instance. Furthermore, the boundaries between various resources, operations, engines, and data storage devices are somewhat arbitrary and specific operations are illustrated within the context of a particular illustrative configuration. Other functional assignments are foreseeable and fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as individual resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as single resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of this disclosure as expressed in the appended claims. Therefore, this specification and drawings should be considered illustrative rather than restrictive.

[0061] The terms “comprising” or “including” are used to indicate the presence of a subsequently claimed feature, but do not preclude the addition of other features. Unless otherwise specifically stated or otherwise understood in the context in which they are used, conditional language such as “may,” “can,” “may,” and “can” is generally intended to convey that certain embodiments include certain features, components, and / or steps that are not included in other embodiments. Therefore, this conditional language is generally not intended to imply that one or more embodiments require features, components, and / or steps in any way, or that one or more embodiments must include logic for determining whether such features, components, and / or steps are included in or performed in any particular embodiment, with or without user input or prompts.

[0062] Although the general outline of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broad scope of embodiments of this disclosure. Where more than one embodiment is disclosed, these embodiments of the subject matter may be referred to individually or collectively herein as the term "invention," this is for convenience only and is not intended to automatically limit the scope of this application to any single disclosure or concept.

[0063] The embodiments illustrated herein are described in detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Therefore, “implementation” is not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims and their full scope.

Claims

1. A method for cross-chip atomic operations, characterized in that, The method includes: On the source computing chip, the atomic operation instructions initiated by the thread are intercepted, and the target memory address corresponding to the atomic operation instructions is determined. In response to determining that the target memory address belongs to a remote computing chip, the atomic operation instruction is forwarded to an atomic operation proxy cache associated with the source computing chip, wherein the atomic operation proxy cache is set in an intermediary layer connecting the source computing chip and the remote computing chip; The atomic operation proxy cache queries based on the target memory address; In response to a successful query, it is determined whether the source computing chip has exclusive modification rights to the target memory address. In response to determining that the source computing chip has the exclusive modification permission, the atomic operation is performed locally by the source computing chip.

2. The method according to claim 1, characterized in that, The query performed by the atomic operation proxy cache based on the target memory address includes: querying the entry table of the atomic operation proxy cache using the cache line address to which the target memory address belongs as an index.

3. The method according to claim 2, characterized in that, Each entry in the entry table includes one or more of the following fields: cache line address, status bit, remote computing chip identifier, atomic operation accumulation value, and pending processing flag.

4. The method according to claim 3, characterized in that, Determining whether the source computing chip has exclusive modification rights to the target memory address includes: when the status bit of the entry corresponding to the target memory address in the atomic operation proxy cache is in an exclusive state or a modified state, it is determined that the source computing chip has exclusive modification rights to the target memory address.

5. The method according to claim 4, characterized in that, The exclusive or modified state indicates that the data corresponding to the target memory address is cached in the local cache of the source computing chip.

6. The method according to claim 4, characterized in that, The method further includes: In response to a query miss or a query hit but the source computing chip does not have the exclusive modification permission, the atomic operation proxy cache buffers the atomic operation instruction; and Multiple atomic operation instructions targeting the same target memory address in the atomic operation proxy cache are merged.

7. The method according to claim 6, characterized in that, The process of merging the multiple atomic operation instructions includes: for multiple combinable atomic operations targeting the same target memory address, accumulating the operands of each atomic operation to the atomic operation accumulation value of the corresponding entry in the atomic operation proxy cache.

8. The method according to claim 6 or 7, characterized in that, The method further includes: When the preset triggering conditions are met, the merged atomic operation request is sent to the remote computing chip for execution.

9. The method according to claim 8, characterized in that, The preset triggering conditions include: The timer expired after the first atomic operation instruction was buffered; The merged operation value reaches the preset threshold; Encountering a memory barrier instruction or synchronization instruction; A request to access the target memory address was received from another computing chip.

10. An apparatus for cross-chip atomic operations, characterized in that, include: Multiple computing chips; An intermediary layer connects the multiple computing chips. The intermediary layer includes an atomic operation proxy cache, which is associated with each of the plurality of computing chips for performing the method according to any one of claims 1 to 9.

11. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, perform the method according to any one of claims 1-9.

Images