On-chip atomic transaction engine

Abstract

The invention relates to on-chip atomic transaction engines. A hardware- assisted distributed memory system can include software-configurable shared memory regions in local memory of each of a plurality of processor cores. Access to these shared memory regions can be via a network of on-chip ATE instances over a private interconnect matrix that connects the on-chip ATE instances together, one ATE instance per core. For example, each ATE instance can issue RPCs, with or without a response, to ATE instances associated with remote processor cores in order to perform operations targeting memory locations controlled by the remote processor cores. Each ATE instance can process (atomically) RPCs received from other ATE instances or generated locally. For some operation types, the ATE instance can itself perform the operation identified in the RPC using dedicated hardware. For other operation types, the ATE instance can interrupt its local processor core to perform the operation.

Description

[0001] This application is a divisional application of patent application no. 201680055397.1, filed on September 21, 2016, entitled "On-Chip Atomic Transaction Engine". TECHNICAL FIELD

[0002] The present disclosure relates generally to distributed shared memory systems, and more particularly to systems and methods for performing atomic transactions on nearby data with dedicated on-chip hardware. BACKGROUND

[0003] As the number of processors in a system continues to grow, maintaining data consistency and hardware scalability are critical. However, traditional shared memory systems (e.g., virtual memory systems) and traditional distributed memory systems fail to meet these requirements, particularly in multi-processor systems where multiple processes are running on the shared memory. Distributed shared memory (DSM) systems, which aim to address both issues, are typically composed of multiple independent processing nodes (each having a local memory module) that talk to each other directly or indirectly using a general-purpose interconnection network. In a typical DSM, shared data is distributed using either a replication strategy or a migration strategy. Replication allows multiple copies of the same data item to reside in different local memories or caches. Once a data item has been replicated in the local memory of a requester, this approach allows the requester to access these data items locally.

[0004] In contrast, migration allows only a single copy of data to exist at any given time. Thus, under a migration strategy, a data item must be moved to the local memory of a requester for exclusive use. Because both replication and migration involve moving data closer to the processor that wants to operate on it, they both incur significant overhead in order to maintain data consistency across all processing nodes. This overhead is primarily due to the complexity of the hardware involved. For example, these systems typically include hardware mechanisms such as synchronization or coherent caches, coherent fabric, and / or snoop logic. In addition, accessing shared data in a DSM typically involves significant latency compared to data accesses targeting local memory. These latencies can result from the steps involved in migrating the target data item to the local memory and updating the corresponding directory structure to reflect the move, or can result from cache flushing and invalidating operations needed to maintain data consistency in the system. SUMMARY

[0005] In various embodiments, the systems described herein can include a hardware- assisted distributed memory system (DSM) in which different software-configurable portions of a distributed shared memory are controlled by respective ones of a plurality of processor cores. In some embodiments, all accesses to these shared memory regions can be made through a network of on-chip atomic transaction engine (ATE) instances. For example, each core can have one ATE instance, and all ATE instances can communicate with each other through a dedicated low-latency interconnect matrix.

[0006] In some embodiments, each processor core or software executing on hardware within each processor pipeline can be configured to determine whether a given operation of a distributed application should be performed by another processor core using a remote procedure call (RPC). For example, if the target of an operation is a location in memory controlled by a different processor, information about the operation can be passed to the local ATE instance. The ATE instance can generate and issue a RPC frame corresponding to the operation with or without a response. The RPC frame can include a descriptor indicating the type of operation, the target address of the operation, and payload information (e.g., the operation’s operands), if applicable. If the target location is controlled by a remote processor core, the local ATE instance can send the RPC frame to the ATE instance associated with the remote processor core, which can place information about the received RPC frame in its local receive queue. If the target location is controlled by a local processor core, the local ATE instance can place information about the RPC in the local receive queue for subsequent processing.

[0007] In at least some embodiments, each ATE instance can retrieve RPCs from its local receive queue and process them (atomically) regardless of whether they were received from other ATE instances or were locally generated. For some operation types, the ATE instance can use dedicated hardware to perform the operation identified in the RPCs it retrieves from its local queue. For example, the ATE can include hardware to perform some relatively simple operations (e.g., read, write, increment, add, or compare and swap) without intervention from the local processor core. For other operation types, the ATE instance can interrupt its local processor core to perform the operation.

[0008] In some embodiments, if the RPC requires a response (e.g., if it is an RPC WR), the operation can be performed by the processor core on behalf of which it was executed can perform one or more other operations after passing the operation to its local ATE instance for processing. At some point (e.g., when the processor core needs the response data), it can issue a "wait for event" instruction in which the event is the return of the RPC response packet from the ATE instance associated with the remote core on which the operation was performed. For example, in some embodiments, the ATE instance can generate an RPC response frame for a received RPC WR, and return it to the originating ATE instance. In some embodiments, when the ATE instance receives the RPC response frame, it can return the response data to its local processor core, or can write it to a memory location from which the local core expects to retrieve it.

[0009] In at least some embodiments, a distributed shared memory system utilizing an ATE network (as described herein) can be more lightweight and less complex than a full cache coherent network. For example, by moving operations closer to the data on which they operate (rather than moving the data close to the processors that initiate the operations), the systems described herein can achieve higher overall performance, and / or can be more flexible than existing DSMs. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a flowchart illustrating one embodiment of a method for utilizing an atomic transaction engine in a distributed computing system.

[0011] Figure 2 is a block diagram illustrating one embodiment of an atomic transaction engine (ATE) instance, in accordance with at least some embodiments.

[0012] Figure 3 is a flowchart illustrating one embodiment of a method for generating and processing RPC frames.

[0013] Figure 4A and 4B is a block diagram illustrating an exemplary RPC frame structure, in accordance with at least some embodiments.

[0014] Figure 5 is a flowchart illustrating one embodiment of a method for using software to determine whether to send an RPC frame to an ATE instance.

[0015] Figure 6 is a flowchart illustrating one embodiment of a method for using hardware to determine whether to send an RPC frame to an ATE instance.

[0016] Figure 7is a flow diagram illustrating one embodiment of a method for processing RPC frames received by an ATE instance.

[0017] Figure 8 is a flow diagram illustrating one embodiment of a method for handling interrupts for received RPC request frames.

[0018] Figure 9 is a flow diagram illustrating one embodiment of a method for implementing broadcast or multicast RPCs in a distributed computing system.

[0019] Figure 10 is a block diagram illustrating use of a full crossbar interconnect in a portion of a distributed computing system in accordance with at least some embodiments.

[0020] Figure 11 is a block diagram illustrating exemplary connectivity with respect to a four-cluster ATE system in accordance with at least some embodiments.

[0021] Figure 12 is a block diagram illustrating a portion of a system including shared multiple banked memory and utilizing an atomic transaction engine instance to manage and perform operations targeting the shared multiple banked memory in accordance with at least some embodiments.

[0022] Figure 13 is illustrated a computing node configured to implement at least some of the methods described herein in accordance with various embodiments.

[0023] While the present disclosure is described herein in exemplification to several embodiments and illustrative drawings, those skilled in the art will recognize that the present disclosure is not limited to the described embodiments or drawings. It is to be understood that the drawings and detailed description thereto are not intended to limit the present disclosure to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims. Any headings used herein are for organizational purposes only and are not intended to limit the scope of the description or the claims. As used herein, the word "may" is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words "include," "including," and "includes" mean including, but not limited to. DETAILED DESCRIPTION

[0024] As noted above, in traditional distributed memory systems, shared data is distributed either using a replication strategy or using a migration strategy. Both replication and migration strategies involve moving data closer to the processor that wants to operate on it, and both strategies incur significant overhead in order to maintain data consistency across all processing nodes. In various embodiments, the hardware- assisted distributed memory systems described herein can include software-configurable shared memory regions in the local memory of each individual core as well as in the main system memory. In these systems, access to the shared memory regions can be through a network of on-chip atomic transaction engine (ATE) instances (e.g., one ATE instance per processor core) and a private interconnect matrix that connects all of the ATE instances together. For operations in which the target address of the operation falls in the local memory of another core, each ATE instance can issue a remote procedure call (RPC) to the other ATE instance. In addition, each ATE instance can process RPCs received from other ATEs. Note that in at least some embodiments, an RPC being processed by an ATE instance can be considered to be a blocking instruction in the sense that the ATE will not accept further instructions (requests for RPCs) until the execution of the current RPC is complete.

[0025] In contrast to traditional DSMs that move shared data (by migration or replication) closer to the processor that wants to operate on it, the systems described herein take the approach of leaving shared data in place and using the processing power already at the location close to the data (either in the form of a local processor core or in the form of an associated ATE instance) to operate on the data. For example, in some embodiments, each ATE instance can police access to the respective portion of the distributed shared memory regardless of which processor wants to operate on data residing in that portion of the distributed shared memory. In some embodiments, each ATE instance can provide a simple instruction set and can perform RPC operations in hardware using these instructions and targeting addresses in the portion of the shared memory policed by that ATE instance. For other operations, the ATE can solicit the help of its local processor core to perform RPC operations targeting addresses in the portion of the shared memory policed by that ATE instance.

[0026] In some embodiments, the methods described herein can alleviate the need to move shared data while still guaranteeing data consistency. In other embodiments, while data coherency can not be guaranteed, the need for hardware-implemented data coherency can be significantly reduced through a combination of constructs provided by the ATE instances and through simple rules that software follows. For example, these constructs can be used to simulate coherency, and if they are used as expected, the system can exhibit coherency. Note that in some embodiments, including those that use distributed shared memory to store variables for synchronization (e.g., in semaphores and mutexes) or for message passing, this hybrid approach can significantly reduce hardware costs (e.g., no coherent cache network is needed) while software complexity is only slightly increased when compared to earlier approaches.

[0027] As described in greater detail herein, if one processor wants to access data in a portion of the distributed shared memory that is controlled by another processor, the requesting processor (or, more specifically, the ATE instance associated with the requesting processor) can send an RPC frame over the dedicated ATE network to the processor that controls the target portion of the distributed shared memory (or, more specifically, the ATE instance associated with the destination processor) rather than moving the desired data closer to the requesting processor. In at least some embodiments, the RPC frame can traverse the dedicated ATE network in a fraction of the time it would take to perform a data migration or to perform a cache flush and invalidate the operations needed to maintain data consistency for the replicated data.

[0028] In some embodiments, when an RPC frame arrives at the destination processor, the ATE instance associated with the destination processor can access the shared memory space controlled by the processor either through the processor's local data memory (DMEM) or through a data cache that accesses the main system memory (e.g., DDR memory). In some cases, the ATE instance can include functionality that allows it to handle some RPCs on its own (e.g., without interrupting the processor). In these cases, while the ATE instance accesses the memory location targeted and performs the operation indicated in the received RPC frame, the processor can do other work (e.g., execute its own instructions or perform its own tasks). In other words, when an ATE receives an RPC request frame, it can be configured to determine whether the RPC operation described in the RPC request frame is something it can handle in hardware (e.g., on its own). If it can, it can access the target address through its own locally accessible portion of the distributed shared memory (sometimes referred to herein as DMEM) or local data cache interface, process the data (as specified in the RPC request frame), and send a response back to the original requester (or its ATE instance), if necessary.

[0029] In some embodiments of the systems described herein, the ATE instances can provide the ability to perform simple operations on shared data at a blistering pace. In these systems, the most recent copy of the shared data need not be moved to a remote core before it can be operated on logically, arithmetically, or otherwise. Instead, the processing power that exists in the vicinity of the data can be used to perform the operation, and the substantial overhead associated with trying to keep caches coherent can be avoided.

[0030] In at least some embodiments, the ATE implementation can define a framework of RPC operations (some of which are implemented in hardware within the ATE instance, while others can be defined by software and / or can require the action of a local processor core). The framework can allow shared data to be operated on before it is sent to the requesting core. In other words, instead of shipping the operands for an operation to the requesting core, a description of the operation can be sent from the requesting core to the core that controls the target data. The data can or can not be required to be returned for various RPCs. In some embodiments, for some RPCs, the data returned as part of the RPC response frame can be stored as the value of a new local variable (e.g., in a special purpose register or in the local memory of the requesting core) for further processing.

[0031] While some of the RPC requests that are automatically handled by the ATE hardware can be relatively simple operations, in some embodiments, the ATE instance can be able to handle more complex operations (e.g., operations related to passing large messages from one core to another, operations that consist of a combination or sequence of relatively simple operations, or other operations for which the ATE instance has included specialized hardware for a particular function). In some embodiments, the ATE instance can know that RPCs with RPC identifiers (RPCIDs) in a particular range of RPCID values are RPCs that it can handle by itself (e.g., without interrupting the processor core). On the other hand, to handle RPCs with RPCID values outside of this range, the ATE instance can need to interrupt the local processor core. In these cases, the local processor core can take an interrupt and process the RPC frame appropriately. In some embodiments, software executing on the local processor can directly issue RPCs and / or can have control over the decision of whether to issue an RPC.

[0032] As noted above, all accesses to the shared address space can go out over the ATE private interconnect. This can help reduce latency for accessing shared data while still keeping hardware costs down (when compared to a cache-coherent based DSM). In such embodiments, the ATE instance network can act as a single access point for the shared address space within the DMEM (or at least the shared portion of the DMEM). For example, a given core can control a particular region of shared memory, and that portion of shared memory should be in its own DMEM. However, all other processor cores can also be able to access that particular region of shared memory. In this example, it can not be desirable to cause a situation where the local ATE instance is acting on a location in the particular region of shared memory on behalf of another core while the local core is directly accessing its DMEM and modifying the data at that location, as this would break the atomicity guarantees that are important in a distributed shared memory system. Thus, in some embodiments, any access to a particular region of shared memory, even by a local processor core that already has direct access to that memory location through its DMEM interface, its data cache interface, or any other interface, can need to request that the ATE instance issue an RPC frame. In this case, the ATE instance can actually send the RPC frame to itself, indicating that it wants to perform the operation. Note that in general, a shared memory region can be in the local DMEM or in main memory (in which case it can be accessed by the ATE through the data cache of the processor core). Thus, in some cases, references to the DMEM in the description herein also apply to the data cache.

[0033] As described in more detail herein, in the event that an ATE instance sends an RPC frame to itself, the ATE instance can recognize the RPC frame as something it needs to process and can queue it in a local RPC receive queue. Note that in at least some embodiments, operations may not be intermingled in the receive queue (e.g., the ATE instance operates on RPC boundaries). In such embodiments, once the ATE instance begins processing an operation for a particular processor core (e.g., the local processor core or a remote processor core), it may not be able to stop and start working on another core until it has completed what it was doing for that particular processor core. In other words, the ATE instance can treat each RPC frame as if it represents an atomic transaction and may not pick up another operation from the RPC receive queue until the previous operation is complete. In this way, the system can ensure that if a local core attempts to access a portion of shared memory that it controls, it cannot overwrite or otherwise alter data that another core may be accessing at the same time.

[0034] Figure 1 The flowchart in FIG. 1 illustrates one embodiment of a method for utilizing an atomic transaction engine in a distributed computing system. As shown at 110, in this example, the method may include assigning control of different portions of a distributed shared memory to each processor core in the distributed computing system. The method may also include distributing data for a distributed application across the different portions of the distributed shared memory and initiating execution of the application, as shown at 120.

[0035] like Figure 1 As shown, the method may include one of the processor cores encountering an operation targeting an address in the distributed shared memory, as shown in 130. If the target address is controlled by that core (shown as a positive exit from 140), the method may include the local ATE instance or that processor core performing the operation (as shown in 150). However, if the target address is not controlled by that core (shown as a negative exit from 140), the method may include the processor core (or its ATE instance) controlling the target portion of the distributed shared memory performing the operation (as shown in 160). In either case, the method may include continuing execution of the distributed application (e.g., without having to wait for the operation to be completed by the remote processor core), as shown in 170. Note that as Figure 1 As shown by the dashed line from 170 to 130 in FIG. 1 , when any processor core in the distributed computing system encounters such an operation, the operations shown in 130 - 160 may be repeated.

[0036] As noted above, in at least some embodiments, an ATE instance can act as a single point of access to a portion of the main system memory that is tagged as shared and allocated to the local core for the purpose of policing. In such embodiments, even if the local core has direct access to the shared region (either because the shared region resides in its local DMEM or through its data cache), all accesses to that region must go through the associated ATE instance. In at least some embodiments, an ATE instance can be given direct access to the data cache (sometimes denoted herein as D$) and DMEM of its local core, which can allow it to operate on shared data without interrupting the local native processor core. In some embodiments, depending on the DMEM and D$ organization, the ATE instance can be allowed to operate on shared data in parallel with the normal execution of its local core, which can improve the overall performance of the system. In some embodiments, there can be an ATE instance for each processor chip (or processor core thereof) in the system, and these ATE instances can be interconnected (and communicate) with each other through a dedicated low-latency interconnect that is separate from the normal bus fabric. In other embodiments, the ATE instances can communicate with each other through the system interconnect.

[0037] Figure 2 The block diagram in FIG. 2 illustrates one embodiment of an ATE instance. Figure 2 The atomic transaction engine (ATE) instance 230 shown in FIG. 2 can implement some or all of the functionality described herein for performing atomic transactions on nearby data in a distributed shared memory system (or managing the performance of these atomic transactions). The ATE instance 230 can include specialized circuitry configured to implement at least some of this functionality on behalf of a given processor core. As Figure 2 As shown in FIG. 2, the frame constructor logic / circuitry 200 within the ATE instance 230 can receive input from the processor's pipeline address decode stage (shown as 202) and / or from the processor's pipeline instruction decode stage (shown as 204). If the pipeline detects that it needs to perform a simple operation on a data item located in shared memory (e.g., a load, store, or increment operation), one subcomponent of the frame constructor logic / circuitry 200 (shown as the auto-RPC frame constructor + address translation and destination core resolution block 210) can issue an RPC. Note that this operation can be managed through the ATE instance 230 regardless of whether the target data item resides on a memory location belonging to the local processor core.

[0038] In some embodiments, if the pipeline detects that it needs to perform a simple operation on a data item located in shared memory, the processor core can provide information about the operation to the ATE instance 230, including the address of the data item on which the operation is to be performed and an identifier or other indication of the core that has control over the portion / region of shared memory that includes the target address. In some cases (e.g., for more complex operations), the processor core can also include information indicating an indexed load or other information specifying the number of elements to load, store, and / or return. The ATE instance 230 can then take this information, generate a descriptor, and appropriately fill in the fields of the descriptor (e.g., based at least in part on the information received from the processor core), and then send the descriptor to the ATE instance associated with the processor core that has control over the portion / region of shared memory that includes the target address. Note that if the portion / region of shared memory that includes the target address is controlled by the local processor core, the ATE instance 230 can actually send the descriptor to itself. However, if the portion / region of shared memory that includes the target address is not controlled by the local processor core, the ATE instance 230 can send the descriptor (and any associated payload information) in an RPC frame over the dedicated ATE network to the ATE instance associated with the processor core that controls that portion / region of shared memory.

[0039] As shown in this example, another subcomponent of the frame constructor logic / circuit 200 (shown as SW RPC payload extraction + frame constructor 212) can be accessed when software executing on the local processor core issues an RPC with a return / response (e.g., RPCWR) or without a return / response (RPCNR), e.g., when the software explicitly includes a remote procedure call. In some embodiments, for example, the processor core can support remote procedure calls with special RPCWR and RPCNR instructions. In this case, the logic / circuit within subcomponent 212 can assemble an RPC frame including an appropriately formed descriptor (as described herein) and any associated payload information (based on, e.g., information in the call by the software and / or based on other inputs to the frame constructor logic / circuit 200), and the ATE instance 230 can send the descriptor to the ATE instance associated with the processor core that has control over the portion / region of shared memory that includes the target address. As noted above, if the portion / region of shared memory that includes the target address is controlled by the local processor core, the ATE instance 230 can actually send the descriptor to itself. However, if the portion / region of shared memory that includes the target address is not controlled by the local processor core, the ATE instance 230 can send the descriptor within an RPC frame to the ATE instance associated with the processor core that has control over the portion / region of shared memory through the dedicated ATE network.

[0040] As shown in this example, the ATE instance 230 can include RPC issue / response logic 214, which can include a response queue and be configured to manage traffic between the ATE instance 230 and the bus interface unit 240, e.g., when the ATE instance 230 sends RPC frames to other ATE instances. In some cases, the ATE instance 230 can send an RPC request frame to another ATE instance and (approximately simultaneously) another portion of the ATE logic / circuit can be processing an RPC request frame that it received from an ATE instance associated with another processor core. For example, the ATE instance 230 can be associated with processor core 0. When attempting to send an RPC request frame to the ATE instance associated with processor core 1, the ATE 230 can be processing an RPC request frame that it previously received from the ATE instance associated with processor core 1. If the command specified in the previously received RPC frame requires a response back to core 1, an RPC response frame will follow the same path as the RPC request frame was sent from the ATE 230 to the ATE instance associated with processor core 1. In other words, both the RPC request frame and the RPC response frame sent from the ATE 230 to the ATE instance associated with processor core 1 can pass through the RPC issue / response logic 214.

[0041] In some embodiments, the RPC issue / response logic 214 can be configured to manage when the last piece of data goes through the RPC issue / response logic 214 and keep the frame together. In some embodiments, the RPC issue / response logic 214 can also be configured to perform some housekeeping functions on the various RPC frames before passing them to the bus interface unit 240. As shown in this example, the RPC issue / response logic 214 can be configured to perform the following functions on the RPC request and response frames: Figure 2 As shown at 234 in FIG. 2B, both RPC request frames and RPC response frames sent from the ATE 230 to other ATE instances are sent via the full crossbar interconnect 250 (private ATE interconnect) (through the master interface 242 of the bus interface unit 240). As shown in FIG. 2B, the full crossbar interconnect 250 is configured to provide a point-to-point connection between each of the ATE instances 230 and each of the other ATE instances 230. Thus, the full crossbar interconnect 250 provides a private interconnect between the ATE instances 230. Figure 2 As shown in FIG. 2B, the bus interface unit 240 can also include a slave interface 244 through which ATE request frames are received (via the full crossbar interconnect 250) from ATE instances associated with other cores for processing by the ATE instance 230. In addition, RPC response frames generated by other cores for RPC requests issued to the ATE instance 230 can also be received through the slave interface 244. The RPC request frames and RPC response frames received through the slave interface 244 can be passed to the RPC receive logic 216 within the ATE instance 230. This RPC receive logic 216 can be configured to determine whether the received RPC frames represent RPC requests or RPC responses and forward them to the appropriate subcomponent of the ATE instance 230. For example, when RPC response frames are received, the response data can be written back into the DMEM at the location where it needs to be written to be returned to the processor core (e.g., through the multiplexer 224 and the DMEM interface 220) so that the processor core can continue. On the other hand, when RPC request frames are received, the RPC receive logic 216 can place the RPC request frames in an RPC receive queue (not shown) for subsequent processing. For example, in some embodiments, there can be a small amount of storage within the local core’s portion of the DMEM in which RPC request frames received from other processor cores (or their associated ATE instances) are buffered before being processed.

[0042] As shown in FIG. 2B, the bus interface unit 240 can also include a slave interface 244 through which ATE request frames are received (via the full crossbar interconnect 250) from ATE instances associated with other cores for processing by the ATE instance 230. In addition, RPC response frames generated by other cores for RPC requests issued to the ATE instance 230 can also be received through the slave interface 244. The RPC request frames and RPC response frames received through the slave interface 244 can be passed to the RPC receive logic 216 within the ATE instance 230. This RPC receive logic 216 can be configured to determine whether the received RPC frames represent RPC requests or RPC responses and forward them to the appropriate subcomponent of the ATE instance 230. For example, when RPC response frames are received, the response data can be written back into the DMEM at the location where it needs to be written to be returned to the processor core (e.g., through the multiplexer 224 and the DMEM interface 220) so that the processor core can continue. On the other hand, when RPC request frames are received, the RPC receive logic 216 can place the RPC request frames in an RPC receive queue (not shown) for subsequent processing. For example, in some embodiments, there can be a small amount of storage within the local core’s portion of the DMEM in which RPC request frames received from other processor cores (or their associated ATE instances) are buffered before being processed. Figure 2As shown in , the ATE instance 230 may include a hardware RPC execution unit 226 that can be configured to perform the operation indicated in the received RPC request frame. For example, in some embodiments, the hardware RPC execution unit 226 can be configured to pull the RPC request frame out of the RPC receive queue, and also begin decoding and processing them, and determine how they should be processed. As described herein, the hardware RPC execution unit 226 may include a processor for performing various relatively simple operations (such as read, write, and increment operations, with and without returns) targeting locally controlled memory locations and data items stored therein. If the hardware RPC execution unit 226 determines that it can process the RPC request frame alone (performing the operation indicated in the RPC request frame), it may do so. For example, the hardware RPC execution unit 226 of the ATE instance 230 may extract the first word of the RPC frame, which may include a descriptor about the RPC frame. As described in more detail below, the descriptor may include information indicating the type of RPC (e.g., the operation to be performed). By processing the descriptor, the hardware RPC execution unit 226 of the ATE 230 can determine whether it can proceed on its own.

[0043] In some embodiments, if the hardware RPC execution unit 226 does not support the operation(s) indicated in the descriptor of the RPC request frame, and / or if the hardware RPC execution unit 226 determines that processing the RPC request frame requires software intervention, it may raise an interrupt to the local processor core, which will then process the RPC request frame. Figure 2 232. More specifically, in some embodiments, in response to such an interrupt, the processor core can access the contents of the RPC request frame by issuing POP and PEEK instructions (shown as input 208 of ATE instance 230). For example, the processor core can issue a POP instruction to pop the RPC request frame from the ATE's receive queue. On the other hand, if the processor core needs to return a response to the RPC request, it can use the PUSH interface (shown as input 206 of ATE instance 230) to send the response data to ATE instance 230 (thereafter, as described above, the RPC publish / response logic 214 within ATE instance 230 can form an appropriate response frame before sending it to the destination ATE instance).

[0044] In some embodiments, when the hardware RPC execution unit 226 of an ATE instance 230 processes a descriptor for an RPC frame to be handled by a local processor core, it can also write control information extracted from the RPC request frame (e.g., from its descriptor) into its own configuration registers 228 (e.g., ahead of the interrupting processor core), and the processor core can (in response to the interrupt) read these registers to determine what it needs to do to handle the RPC. For example, the information in the configuration registers can indicate how many words the processor core needs to pop from the receive queue to read the entire RPC request frame, can indicate whether the processor core needs to send a response back to the originator of the request, and (if so) can indicate how many words the processor core needs to send back in a corresponding RPC response frame. In some embodiments, while the processor core is processing the RPC request frame (e.g., while it is performing one or more operations indicated in the RPC request frame), and as it produces response data by performing these operations, the processor core can use the RPC response PUSH interface to move the response data to the ATE instance 230 for subsequent inclusion in an RPC response frame.

[0045] As Figure 2As shown in FIG, logic / circuitry (220), shown as an interface with the DMEM, manages read and write traffic between the ATE instance 230 and the locally accessible portion of the DMEM (shown as 236). For example, the interface can be used to read and write to portions of the distributed shared memory controlled by the local processor core and / or other memory locations accessible to the local processor core. Similarly, logic / circuitry (222), shown as an interface with the data cache, manages read and write traffic between the ATE instance 230 and the data cache of the local processor core (shown as 238). In some embodiments, the (one or more) data items in the distributed shared memory to be operated on by the ATE instance 230 or the local processor core can be found in one of these two places. As noted above, in some embodiments, the RPC receive queue can also reside in the DMEM (not shown). In such an embodiment, all access to the RPC receive queue can be made through the DMEM interface 220. Note that placing the RPC receive queue in the DMEM (which can include a relatively large amount of local memory) can allow software to configure the size of the RPC receive queue within that block of local memory based on expectations or workload. Allowing ATE instance 230 to buffer a large number of RPC frames can prevent the entire network from being stalled while ATE instance 230 is processing RPC request frames. In some embodiments, the size of the RPC receive queue can be dynamically changed based on workload. In other embodiments, software can allocate a specific amount of space for the RPC receive queue and communicate this to ATE instance 230 so that ATE instance 230 knows how much memory it has to work with. Note again that in some embodiments, the interface between DMEM and ATE instance 230 can include a relatively fast interconnect. For example, in some embodiments, there can be a direct connection between the ATE instance and DMEM.

[0046] In one embodiment, the atomic transaction engine configuration registers (which may also be referred to as control and status registers) may include one or more of the registers shown in Table 1 below. In other embodiments, the atomic transaction engine may include more, fewer, or different control and status registers. Note that in different embodiments, the ATE control and status registers may be arranged in any order (and may be placed contiguously or non-contiguously in the address space).

[0047]

[0048] Table 1. Example ATE Control and Status Registers

[0049] In some embodiments, the ATE control and status registers can also include a field for specifying a starting address in DMEM for an ATE receive queue, a field for specifying a size of the receive queue (e.g., in words), and / or a field for specifying a number of words in the current receive queue (e.g., one field can specify a number of words present in the receive queue, and another field can specify a number of words that have been pre-fetched from the receive queue), any or all of which can be software configurable to control various ATE functions.

[0050] In some embodiments, the ATE can pre-fetch one or more words (e.g., in one embodiment, up to two words) from the receive queue at any given time. In some embodiments, the total number of buffered RPC words can be calculated as the number of words in the receive queue plus the number of words that have been pre-fetched from the receive queue. In some embodiments, the size of the short RPC receive buffer can be assumed to be 8 words, which is the maximum size of an RPC response frame.

[0051] In at least some embodiments, when software creates a descriptor for use with RPC instructions that do not return (RPCNR) and RPC instructions that do return (RPCWR), it can fill various fields of the descriptor to specify an RPC type identifier (sometimes referred to herein as an RPC ID), a source core identifier (SRC ID), a destination core identifier (or DEST ID), a length of the payload (e.g., in words), and / or a length of the return payload (e.g., words), if applicable. In some embodiments, the SRC ID field can be filled by the hardware (e.g., automatically).

[0052] In some embodiments, certain RPC ID values (e.g., 0-15 in one embodiment) can be reserved for RPCs that are implemented in hardware and do not require software support. As noted above, a processor core can be allowed to send an RPC to itself (e.g., the DEST ID field can indicate the local processor core). In fact, in some embodiments, when operating on shared data (e.g., anywhere within the distributed shared memory), software must use the RPC mechanism described above to perform the operation (rather than directly loading and storing) in order to avoid coherency issues.

[0053] In various embodiments, the interrupt register can include fields representing one or more of an interrupt that occurs when the ATE pulls an RPC frame from the ATE receive queue that needs to be serviced by software, an interrupt that occurs when both the ATE receive queue and the pre-fetch buffer become full, or an interrupt that occurs when both the ATE receive queue and the pre-fetch buffer become empty. In some embodiments, these interrupts can be cleared by writing to these fields.

[0054] The interrupt enable register can include fields representing one or more of: a bit to enable an interrupt to occur when the ATE pulls an RPC frame from the ATE receive queue that needs to be serviced by software, a bit to enable an interrupt to occur when both the ATE receive queue and the prefetch buffer become full, or a bit to enable an interrupt to occur when both the ATE receive queue and the prefetch buffer become empty. In some embodiments, the corresponding interrupt can be enabled when one of these bits is set, and can be masked when one of these bits is cleared.

[0055] In some embodiments, when software writes a field in the interrupt register representing an interrupt to occur when the ATE pulls an RPC frame from the ATE receive queue that needs to be serviced by software, the hardware can check the next word in the receive queue as an RPC descriptor. If the descriptor is a software-based RPC, the bit can be set in the next cycle and can need to be explicitly cleared by software when it is being processed. If the interrupt is masked, the software can need to poll the bit to determine if a software RPC is available. In this example, the software can write a 1 to the field in the interrupt register to clear the interrupt. In this example, when software writes a field representing an interrupt to occur when both the ATE receive queue and the prefetch buffer become full, the hardware can again check the receive queue status. If the queue is still full, the interrupt can be set again in the next cycle. In some embodiments, the software can write a 1 to the field to clear the interrupt. In this example, when software writes a field representing an interrupt to occur when both the ATE receive queue and the prefetch buffer become empty, the hardware can again check the receive queue status. If the queue is still empty, the interrupt can be set again in the next cycle. In some embodiments, the software can write a 1 to the field to clear the interrupt.

[0056] In one example embodiment, from the perspective of each processor core (e.g., each general purpose or special purpose processor core), the data space can be divided into three high-level regions:

[0057] • a region that is a cacheable main system memory space

[0058] • a region that is a core-specific private space

[0059] • a region that corresponds to an external peripheral space

[0060] In this example, the core-specific private space can be unique to each processor core. It can not be externally visible, nor accessible by components other than its corresponding processor core and the corresponding ATE instance.

[0061] As noted above, the ATE instance associated with each processor core can act as a gateway for all shared memory accesses in the system. In various embodiments, it can receive RPC send requests from its local core, construct corresponding RPC frames, and send them to various remote cores (or back to the local core) for processing. As described herein, in some embodiments, access to shared memory regions can be performed by the core in the form of explicit RPC instructions or implicitly by the core pipeline based on software-configurable shared address decoding logic / circuitry. For example, in some embodiments, once the address decode stage of the local core determines that an operand is a shared memory location, an RPC can be automatically issued by the local core. In some embodiments, these automatically issued RPC operations can be limited to relatively simple instructions such as LOAD, STORE, ADD, INC, and Compare-and-Swap. In some embodiments, the ATE instance can provide a small instruction set that can be used by software to describe the structure of an RPC frame and its identifier (e.g., its RPC ID) so that the ATE hardware itself can identify and execute the RPC (rather than issuing an RPC interrupt to the local processor core and requesting the local processor core to execute the RPC in software). In some embodiments, this can result in reduced total latency in the system.

[0062] In some embodiments, RPCs can also be explicitly issued under software control by using one of the processor core's RPC instructions. The type of operation in these RPCs can range from simple reads or writes to message passing in different embodiments. Prior to issuing the explicit RPC instruction, software can assemble the RPC payload in its local DMEM or general-purpose / special-purpose registers, and can then pass a pointer to this location along with other control information as part of the RPC instruction to the local ATE instance. The RPC instruction can include a descriptor containing control information such as the RPC ID or operation identifier, the number of words being sent, and / or the number of words expected to be returned in the response frame, the destination core, and a pointer to the earlier constructed RPC frame payload. The local ATE instance can then assemble the actual RPC frame by extracting the payload data from the DMEM, and can send the frame over the private ATE interconnect network.

[0063] As noted above, an RPC requested by a processor core can or can not expect the remote processor core that performs the requested operation to return data (e.g., it can be an RPCWR or an RPCNR). For example, a processor core can issue an RPCNR to tell another processor core to perform some operation for which the initiating requesting core does not need (or does not expect) a return value. Such an operation can be used, for example, to create a check-in-point, where one of the processor cores in the system acts as a master core to which the other cores (e.g., acting as slave cores) can periodically check in. In this example, each of the slave cores can issue an RPC to the master core indicating that it has reached a check-in point when it reaches that point. The RPC can perform a simple operation such as writing to a particular address location, or incrementing the value of an address location by one to indicate how many of the cores have reached the synchronization point. In this example, after sending the RPC to the master core, the slave core can wait at that point (from when it has reached the synchronization point). When all of the slave cores have checked in, the master core can send out another RPCNR (to all of the slave cores) to indicate, for example, that the slave cores should resume execution or initiate execution of a new set of operations.

[0064] In some embodiments, when the processor core that initiated the request needs (or expects to receive) one or more return values, the processor core can issue an RPCWR to tell another processor core to perform some operation on shared data. In one example, the core that initiated the request can want to perform a read of a memory location controlled by another core. In this case, the core that initiated the request can issue an RPCWR to the other core, which will read the memory location and send the read data back to the core that initiated the request in a response RPC frame. In another example, an RPCWR can be used to create a semaphore. In this example, if a core wants to access a shared resource, it can issue an RPCWR to a remote core that controls a shared memory location (a semaphore location) that indicates whether the shared resource is available, specifying a compare-and-swap (CAS) type operation. As described herein, an ATE instance can include circuitry to perform such CAS instructions in hardware. In this example, the core can issue a particular value (e.g., a value of 1) to the remote core that controls the shared memory location that indicates whether the shared resource is available. If the value of the semaphore location indicates that the shared resource is available, the remote core can return a value (e.g., a value of 0 or some other value that indicates that the shared resource is available). If the remote core returns a value that indicates that the shared resource is available, the core that initiated the request can proceed. In this example, because the CAS instruction was used to update the shared memory location controlled by the remote core, any other core that wants to access the shared resource will have to wait (e.g., because the semaphore location has a value that indicates that the resource is currently unavailable (i.e., it is being used)).

[0065] In some embodiments, if the processor that initiated the request expects a return (e.g., if the RPC is an RPCWR), the ATE instance associated with the processor that initiated the request can also capture the RPC response frame coming back from the remote processor core and either directly deliver it to the pipeline of the local core (e.g., if it was stalled waiting for that data) or can move the response data to a location in the local DMEM that the core specified when it issued the RPC.

[0066] In some embodiments, if a processor core issues an RPC with a return, it can stall at that point, waiting for the response. However, other embodiments can utilize a "wait for event" construct to avoid stalling at that point. In such embodiments, after issuing an RPC transaction, if the requesting core needs to wait for a response, there can be a finite amount of time between issuing the RPC and retrieving the response (e.g., the time it takes for the local ATE instance to send the RPC over the ATE network to another ATE instance, the remote ATE instance to perform the operation indicated in the RPC, and the remote ATE instance to return the response to the local ATE instance). In some embodiments, rather than waiting for all of this to happen, the requesting core can execute one or more other instructions (e.g., instructions that do not depend on the response data) while this is happening. At some later point in time (e.g., at the point in time when the requesting core needs the response), the requesting core can issue a "wait for event" instruction (e.g., in software executing on the requesting core) and can specify an event ID associated with a local or remote ATE instance. In this example, once the "wait for event" instruction has been issued, it can create a barrier point that the core will not proceed past until the event being waited for has occurred. In some embodiments, the event being waited for can be the remote ATE having returned response data for a previous RPC WR or the local ATE having written the response data back to the location pointed to by the requesting core at the time the RPC was issued.

[0067] Figure 3 The flowchart in FIG. 3 illustrates one embodiment of a method for generating and processing RPC frames. As shown at 310, in this example, the method can include a processor pipeline detecting an operation targeting an address in a distributed shared memory and sending information about the operation to its local ATE instance. The method can include the local ATE instance generating an RPC frame including a description of the operation and, if applicable, payload information, as shown in 320. Note that depending on the operation, the RPC frame can specify an RPC with a return or an RPC without a return.

[0068] As shown in this example, if the target address is controlled by this core (shown as a positive exit from 330), the method can include the local ATE instance placing information about the operation in a local receive queue, as shown in 335. The method can also include the local processor core (or its ATE instance) processing the RPC frame as an atomic transaction, as shown in 340, after which execution of the application including the operation proceeds, as shown in 350. If, on the other hand, the target address is not controlled by this core (shown as a negative exit from 330), the method can include the local ATE instance sending the RPC frame to the ATE instance of the processor that controls the target address for processing, as shown in 355.

[0069] As described herein, in some embodiments, an RPC frame can be sent from a local ATE instance to another ATE instance via a dedicated ATE interconnect. If the RPC frame is for an RPC with a return (shown as a positive exit from 360), the method can include the local processor core issuing a "wait for event" barrier while awaiting the RPC response to be received by its ATE instance, as shown in 365. Note, however, that if the local processor core has other work to do that does not depend on the response, it can perform one or more operations between the time the RPC frame is sent to the other ATE instance and the time it issues the "wait for event" barrier. As shown in this example, once the wait event occurs, the method can include the local processor core retrieving the RPC response, as shown in 370, after which execution of the application including the operation proceeds, as shown in 350. Note, as shown in 365, that the local processor core can issue the "wait for event" barrier before the RPC frame is sent to the other ATE instance. Figure 3 As shown in 360, if the RPC frame is for an RPC without a return (shown as a negative exit from 360), no response can need to be awaited before execution proceeds at 350.

[0070] Figure 4A and 4B An exemplary RPC frame structure is illustrated in accordance with at least some embodiments. More specifically, Figure 4A An exemplary RPC request frame 400 is illustrated, Figure 4BExemplary RPC response frames 450 are illustrated. In these examples, the request frame 400 includes an RPC descriptor 402 (which can include any or all of the exemplary descriptor information described herein and / or other information) and a data start address 404 (which can identify a start address of the target(s) of the operation), and can (optionally) also include one or more payload data elements (shown as 404-410). Note that in some cases, one or more of the payload data elements can represent an address. In these examples, the response frame 420 includes at least one response data element (shown as 422), and can (optionally) include one or more additional response data elements (shown as 424-426).

[0071] As noted previously, some of the RPCs that can be handled directly by the ATE instance itself can be hard-coded (e.g., implemented in dedicated circuitry of the ATE instance), while others can be programmed by the local processor core using a set of configuration registers and a small instruction set provided by the ATE instance. In some embodiments, this can allow relatively simple software routines to be implemented, such that they are handled entirely by the local ATE instance without the need for interrupting the core. This can in turn reduce the overall latency of the system, and increase the overall system throughput (e.g., MIPS). In one exemplary embodiment, the RPC commands that can be implemented in hardware in the ATE instance can include those shown in Table 2 below. In other embodiments, more, fewer, or different RPC commands can be implemented in hardware in the ATE instance.

[0072] Command RPC ID Description DataRead 0 Read data from specified memory address DataWrite 1 Write data to specified memory address AtomicAdd 2 Add value to memory AtomicAddRtn 3 Add value to memory with return CmpAndSwap 4 Atomic compare and swap

[0073] Table 2. Exemplary hardware-implemented ATE RPC commands

[0074] Note that the target addresses of these operations can be in the data cache, or in the data memory (DMEM). In some embodiments, the addresses sent as part of the RPC can be padded with zeros to form appropriately-sized addresses, regardless of whether they are located in the data cache or in the data memory space. Some of these commands can return data to the RPC initiator (the processor core that issued the RPC command). Those response data values can be written into the payload area of the corresponding RPC response frame, starting with the first word of the payload area. According to at least some embodiments, the exemplary commands listed above are described in more detail below.

[0075] In this example, the DataRead command (RPC ID = 0) can be used to read data from memory and return one or more data words. In some embodiments, a DataRead type RPC can be sent either as an RPC WR or as a shorter RPC WRS. The payload (i.e., operand) for this command can include one or more words specifying the target address, respectively. The return payload for this command can include the number of words specified in its descriptor. For example, in one embodiment, the return payload can include 1-8 words for an RPC WR, and 1-6 words for a shorter RPC WRS. Note that the ATE can automatically increment the memory address after each access, with the increment in address depending on the size of the payload.

[0076] In this example, the DataWrite command (RPC ID = 1) can be used to perform sequential data memory writes. In some embodiments, a DataWrite type RPC can be sent either as an RPC NR or as a shorter RPC NRS. The payload (i.e., operand) for this command can include one or more words specifying the target address and up to the number of words specified in its descriptor one or more additional words containing the data to be written. For example, in one embodiment, the payload can include 3-8 words for an RPC NR command to write 1-6 data items, or 3-6 words for a shorter RPC NRS command to write 1-4 data items. Since there is no return value for this command, the descriptor can indicate a return payload length of 0. Note that the ATE can automatically increment the memory address after each access, with the increment in address depending on the size of the payload.

[0077] In this example, the AtomicAdd command (RPCID=2) can be used to change a value in a distributed shared memory (e.g., DMEM) or a data cache (D$) using an atomic addition or subtraction operation. In other words, the command can be used to read a data value from a target memory location, add a 32-bit value to the read content, and write the resulting value back to the same memory location. In some embodiments, up to six such modified values ​​can be written to memory using this command. In some embodiments, an RPC of type AtomicAdd can be sent either as an RPCNR or as a shorter RPCNRS. The payload (i.e., operand) for the command can include one or more words specifying the target address and one or more additional words (containing the data to be written) up to the number of words specified in its descriptor. For example, in one embodiment, for an RPCNR command that writes 1-6 data items, the payload can include 3-8 words, or for a shorter RPCNRS command that writes 1-4 data items, the payload can include 3-6 words. Because there is no return value for this command, the descriptor can indicate that the return payload length is 0. Note that ATE may automatically increment the memory address after each access, where the amount by which the address is incremented depends on the size of the payload.

[0078] In this example, the AtomicAddRtn command (RPCID = 3) can be used to change a value in a distributed shared memory (e.g., DMEM) or data cache (D$) using an atomic addition or subtraction operation. In other words, this command can be used to read a data value from a target memory location, add a 32-bit value to the contents read, and write the resulting value back to the same memory location. In some embodiments, the payload can include up to six addend values, and the first return value can be written to the same address as the first word of the payload. In some embodiments, an AtomicAddRtn type of RPC can be sent either as an RPCWR or as a shorter RPCWRS. The payload (i.e., the operand) for this command can include one or more words specifying the target address and up to the number of words specified in its descriptor of additional words containing the data to be written. For example, in one embodiment, the payload can include 3-8 words for an RPCNR command to write 1-6 data items, or 3-6 words for a shorter RPCNRS command to write 1-4 data items. In some embodiments, the return payload length can be equal to the number of addend values in the payload, which is also the number of addition result values returned. In such embodiments, the descriptor can indicate that the return payload has two fewer elements than the payload. Note that the ATE can automatically increment the memory address after each access, where the increment of the address depends on the size of the payload.

[0079] In this example, the CmpAndSwap command (RPCID = 4) can be used to compare a memory value to a payload value, and then perform a swap operation based on the comparison of the memory value to the specified comparison value. The entire operation can be considered atomic. For example, the CmpAndSwap command can be used to perform an operation similar to that illustrated by the following pseudo-code:

[0080]

[0081] In some embodiments, a CmpAndSwp type of RPC can be sent either as an RPCWR or as a shorter RPCWRS. The payload (i.e., the operands) for this command can include one or more words specifying a target address, followed by one or more compare-and-swap value pairs (for each compare-and-swap value pair, a compare value occupies one word and a swap value occupies one word). For example, if only a single compare-and-swap value pair is included in the payload, the payload can include four words, but if three compare-and-swap value pairs are included in the payload, the payload can include eight words. In this example, the payload can include 4, 6, or 8 words for an RPCWR command that performs 1, 2, or 3 compare-and-swap operations, or 4 or 6 words for a shorter RPCWRS command that performs 1 or 2 compare-and-swap operations. In this example, the return payload length can be equal to the number of compare-and-swap value pairs (e.g., 1, 2, or 3 for an RPCWR command, or 1 or 2 for a shorter RPCWRS command). Note that the ATE can automatically increment the memory address after each compare-and-swap operation, where the increment amount of the address depends on the size of the payload.

[0082] In different embodiments, there can be different ways of determining whether (or when) to issue an RPC frame to perform an atomic transaction, with some ways being implemented primarily in hardware and others being implemented (or in conjunction with) software executing on a given node. Figure 5 The flowchart in FIG. 5 illustrates one embodiment of a method for using software to determine whether to send an RPC frame to an ATE instance. As shown at 510, in this example, the method can include software executing on a given processor core in a distributed computing system encountering an operation targeting an address in memory. If the software determines that the target address is the address of a location in the distributed shared memory (shown as a positive exit from 520), the method can include the software forming a descriptor and (optionally) payload data for a corresponding RPC targeting the shared memory, as shown in 530. In this case, the method can also include the software making an explicit RPC call (with or without a return) that includes the descriptor and (if applicable) a pointer to the payload data in memory, as shown in 540.

[0083] On the other hand, if the software determines that the target address is not an address of a location in the distributed shared memory (shown as a negative exit from 520), the method can include the software executing an operation on the given processor core, as shown in 525. In either case, the method can include the software continuing to execute on the given processor core (e.g., after the processor core executes the operation or while the ATE is processing the RPC), as shown in 550. As shown by the positive exit from 560 back to 520, in this example, if another operation targets a location in memory (shown as a positive exit from 560), the method can include repeating the operations shown in 520-550 for each additional such operation.

[0084] As Figure 5 shown and described above, in some embodiments, the software executing on a given processor core can be able to determine whether it should make an RPC call to perform an operation on data at a shared memory location. In other embodiments, specialized hardware within the processor core can be used to make (or support) such determinations. For example, in some embodiments, a translation table structure can be built within shared region decoder logic / circuitry in the processor core, and the pipeline logic / circuitry can be able to determine whether an instruction is a shared region transaction. If so, it can be handed off to the local ATE instance for processing, rather than being facilitated in the pipeline. In other words, in some embodiments, an address decode table can be built inside each processor core (e.g., within the address decode stage) that determines whether an operation (e.g., a load or store) targets the distributed shared memory. If so, the pipeline can automatically initiate an ATE transaction by feeding various control parameters to the local ATE instance in a corresponding RPC frame, and the local ATE instance can send it to a remote ATE instance (or itself) for processing. Regardless of whether software or hardware is used to determine whether to issue an RPC, using the network of ATE instances to process the RPC can greatly reduce the required software intervention, at least for some simpler operations performed by the application.

[0085] Figure 6The flowchart in FIG. 6 illustrates one embodiment of a method for using hardware to determine whether to send RPC frames to ATE instances. As shown at 610, in this example, the method can include software executing on a given processor core in a distributed computing system encountering an operation targeting an address in memory. The method can include an address decode stage within the processor core comparing the target address to entries in an address decode table, as shown in 620. If the target address is an address of a location in the distributed shared memory (shown as a positive exit from 630), the method can include the processor pipeline automatically supplying operation parameters to an ATE instance within the processor core, initiating an ATE transaction in 640.

[0086] On the other hand, if the target address is not an address of a location in the distributed shared memory (shown as a negative exit from 630), the method can include the given processor core executing the operation, as shown in 635. In either case, the method can include continuing execution on the given processor core (e.g., after the processor core executes the operation or while the ATE is processing the RPC), as shown in 650. As shown by the positive exit from 660 back to 620, in this example, if another operation targets an address in memory (shown as a positive exit from 660), the method can include repeating the operations shown in 620-650 for each additional such operation.

[0087] As noted earlier, in addition to sending RPC frames to other ATE instances, each ATE instance can also service RPC frames issued by other cores that want to access the shared memory space managed by that particular ATE instance. In some embodiments, these RPCs can be received from the interface via the ATE instance's interconnect, and can be queued for processing in a receive buffer (e.g., in the ATE's own internal buffer space or in a hardware-managed FIFO formed within the DMEM as specified by the software by writing to a configuration register of the ATE). The ATE instance can then pop RPC frames from this receive queue for processing (e.g., in the order in which they were received, or according to some other priority policy or scheme). As noted above, the ATE instance can also send RPC frames to other ATE instances in response to RPCs received from other cores. Figure 4AAs shown in the middle and as described herein, each RPC request frame can include a descriptor header followed by a target memory location address and any optional payload words. The descriptor can contain information about the RPC ID, the size of the payload data, the amount of data to be returned (if any), etc. As described herein, the ATE can use the RPC ID information in the descriptor to determine whether it can handle and send a response to the received RPC (if requested) by itself or whether it needs to stop and interrupt the local processor core to handle the RPC.

[0088] In some embodiments, the ATE instance can monitor its receive queue (which is sometimes referred to herein as an "ATE receive queue" or "RPC receive queue") and when an RCP appears at the top of the queue, it can be immediately pulled off the queue and processed (e.g., executed). In some embodiments, the ATE instance can determine based on the RPC ID whether it is an RPC that can be executed by hardware within the ATE instance or whether software executing on a processor core must execute the RPC. For example, in one embodiment, RPC IDs within the range 0-15 can be reserved for operations performed with hardware. If the operation is such a hardware-based RPC, it can be executed by the ATE instance, which can return any response data to the ATE instance of the core that initiated the request (as needed).

[0089] In this example, if the RPC is not a hardware RPC (e.g., if its RPC ID > 15), the ATE can signal an interrupt to the local processor core. The interrupt service routine for the processor core can read the RPC descriptor register and, based on the value of the RPC ID, can call the appropriate interrupt handler. As mentioned previously, software can be able to pass operands with the RPC. These operands can be accessed by the receiving processor-side software by using PEEK and POP instructions on the ATE receive queue. However, the software can need to be careful not to pop more data from the queue than was actually sent. As noted above, the value in the RPC descriptor register can indicate the size of the payload.

[0090] In at least some embodiments, if the RPC requires data to be returned (e.g., if the RPC is RCPWR), the software can generate the response data and pass it to the hardware by using a PUSH instruction on the ATE response queue within the RPC issue / response logic. Again, it is reiterated here that the software can need to be careful not to push more data than the maximum amount of data allowed to be returned. For example, the ATE instance can generate a response back to the originating processor core once the number of words pushed into the ATE receive queue matches the return payload length value specified in the descriptor. In some embodiments, e.g., for best performance, the software can check for and execute any other software-based RPCs before exiting the RPC software interrupt service routine. In such embodiments, the software can exit the RPC software interrupt service routine when a hardware-based RPC is present, or when the queue is empty.

[0091] In some embodiments, if the core is interrupted to process the RPC, it can notify the ATE instance that it has finished processing the RPC by clearing the RPC interrupt status register. The ATE instance can then send the response data to the core that initiated the request in the form of an RPC response frame, such as the RPC response frame shown in Figure 4B

[0092] Figure 7 The flowchart in FIG. 7 illustrates one embodiment of a method for processing RPC frames received by an ATE instance. As shown at 710, in this example, the method can include the ATE instance of a given core in a distributed computing system receiving an ATE frame from another ATE instance or from the given core itself. If the ATE frame is an RPC request frame (shown as a positive exit from 720), the method can include the ATE instance placing the RPC request frame in a receive queue, as shown in 730. At some later point, the method can include the RPC execution unit of the ATE instance retrieving the RPC request frame from the receive queue and examining its descriptor (or a portion of its descriptor, such as the first word), as shown in 750.

[0093] If the ATE instance determines (based on the examined descriptor or portion thereof) that it can process the RPC request itself (shown as a positive exit from 760), the method can include the RPC execution unit processing the request as an atomic transaction and generating and returning an RPC response in the ATE frame, if applicable, as shown in 780. However, if the ATE instance determines (based on the examined descriptor or portion thereof) that it cannot process the RPC request itself (shown as a negative exit from 760), the method can include the ATE instance placing information about the request into one or more configuration registers within the ATE instance and issuing an interrupt to a processor core to process the RPC request, as shown in 770.​Figure 8 The flowchart in FIG. 8 illustrates one embodiment of a method for handling such an interrupt. In this example, if the ATE frame is not an RPC request (e.g., if it is an RPC response frame received from another ATE instance to which an RPC request frame was previously sent), the method can include the ATE instance writing the returned response to the distributed shared memory for subsequent retrieval by a processor core. This is illustrated with the negative exit from 720 and element 740 in FIG. 8. Figure 7

[0094] Figure 8 The flowchart in FIG. 8 illustrates one embodiment of a method for handling such an interrupt. In this example, if the ATE frame is not an RPC request (e.g., if it is an RPC response frame received from another ATE instance to which an RPC request frame was previously sent), the method can include the ATE instance writing the returned response to the distributed shared memory for subsequent retrieval by a processor core. This is illustrated with the negative exit from 720 and element 740 in FIG. 8.

[0095] As shown in this example, if the RPC request is a no-return RPC request (shown as exiting negatively from 850), the processing of this RCP request can be complete, as shown in 880. However, if the RPC request is a return RPC request (shown as exiting positively from 850), the method can include the processor core using the PUSH interface to move the response data to the ATE instance, as shown in 860. In this case, the method can also include the ATE instance assembling a response frame and sending it to the ATE instance from which the RPC request was received, as shown in 870, after which the processing of this RCP request can be complete, as shown in 880.

[0096] ​In some embodiments, a broadcast RPC operation or a multicast RPC operation can be issued such that the same message or operation request can be sent to multiple processor cores at the same time (e.g., for synchronization purposes, as a semaphore, or to initiate the start of a routine on multiple processor cores), after which they can be processed by each recipient substantially in parallel. While there can be no guarantee that all receiving ATE instances will see and process these RPCs at exactly the same time, in at least some embodiments, it can be guaranteed that they will all eventually see and process the RPCs. In one example, a core issuing a single RPC can configure the RPC request such that once it reaches the local ATE instance, the ATE instance can issue the corresponding RPC frame to every processor core in the system or a subset of the processor cores in the system. Using this approach, rather than the software executing on the processor core having to issue multiple individual RPCs, the local ATE instance can automatically handle sending the broadcast or multicast operation, which can reduce the overhead incurred by the processor core.

[0097] In some embodiments, a broadcast RPC issued by a processor core can include a broadcast indicator that is interpreted by the local ATE instance. When such a request is issued by a processor core, the ATE instance will issue the same RPC to all of the processor cores in the system. In some embodiments, a multicast RPC issued by a processor core can include an indication that the RPC should be multicast and an identifier of the processor cores to which the RPC will be sent. For example, the request can include a bit vector, a pointer to a list of recipient processor cores, or some other indicator of the target recipients, and the local ATE instance can interpret this information and send the RPC to the processor cores designated as recipients of the request. In embodiments that use a bit vector to identify recipient cores, the local ATE instance can inspect the bit vector to determine which cores should receive the RPC, and as it works through the bit vector, it can send the RPC to each core (e.g., one at a time). In other embodiments, the ATE instance can first determine all of the recipient processor cores, and then it can send the RPC frame to them all at approximately the same time. For example, in some embodiments, the system can provide broadcast functionality in the interconnect such that the RPC frame can be broadcast in a branch or waterfall fashion.

[0098] Figure 9The flowchart in illustrates one embodiment of a method for implementing broadcast or multicast RPC in a distributed computing system. As shown at 910, in this example, the method can include software executing on a given processor core encountering an operation targeting multiple addresses in a distributed shared memory, where each address is associated with (or controlled by) a different core. For example, the operation can be a write operation that acts as a semaphore or another type of message that will be broadcast to all processing nodes in the distributed computing system or multicast to some (but not all) processing nodes in the distributed computing system. The method can include the software (or hardware within the processor pipeline) sending information about the operation to a local ATE instance, including an indication of which processor cores should receive the corresponding RPC frame, as shown in 920. For example, the information provided to the local ATE instance can include an indication that the RPC should be broadcast to all processor cores or an indication of the target processor cores for the multicast operation (e.g., a bit vector representing the target cores, a pointer to a list structure whose elements identify the target cores, or another identifier of the recipient processor cores).

[0099] like Figure 9 As shown in , the method may include the local ATE instance interpreting the received information and generating an RPC frame including a description of the operation and any applicable payload information, as shown in 930. If the received information indicates that the RPC should be broadcast to all processor cores in the distributed computing system (shown as a positive exit from 940), the method may include the local ATE instance sending the RPC frame to ATE instances of all processor cores in the system (e.g., serially or approximately simultaneously), as shown in 960. However, if the RPC request is intended to be multicast to a subset of processor cores in the distributed computing system, rather than to all of them (shown as a negative exit from 940), the method may include the local ATE instance sending the RPC frame to ATE instances of the identified cores (e.g., serially or approximately simultaneously, if broadcasting is supported in the interconnect), as shown in 950. Note that following the broadcast or multicast of the RPC frame, execution on a given processor (i.e., the processor core that issued the broadcast or multicast RPC) may continue while ATE instances of other cores process the RPC frame sent by the given processor core. In some cases, if the broadcast or multicast RPC is an RPC with a return, the given processor core may then receive (or retrieve from DMEM) responses from each processor core that received the RPC frame (which may include itself in some cases) and examine those responses.

[0100] In at least some embodiments, unlike in conventional DSMs, the ATE subsystem described herein can combine message passing / inter-process communication between cores with shared data access. For example, to pass a message to a remote core, a local core can use an RPC construct to send the message as the payload for the RPC (for short messages destined for the remote core). Alternatively, the local core can use one or more ATE hardware write RPCs to assemble the message in the remote core's shared memory area, after which it can issue an ATE message notify RPC (which may include the message's address and size) to notify the remote core that the message is ready to be read.

[0101] Note that in various embodiments, the hardware and software mechanisms described herein for utilizing the atomic transaction engine can be applied to systems implemented on a single chip and including multiple processors (each having one or more processor cores) on the same chip. However, they can be used in systems including multiple processors (each having one or more processor cores) on different processor chips on a single board or on multiple boards in the system. In any of these cases, each processor core of each processor chip can be associated with a different ATE instance, and all ATE instances can be interconnected via a dedicated ATE network (which can be implemented as a private low-latency bus).

[0102] In some embodiments, the ATE network may include one or more clusters of interconnect matrix components, where each cluster may implement a full crossbar interconnect (such as Figure 2 Each matrix can connect several physically close ATEs together (e.g., ATE instances corresponding to processor cores on the same chip) and can also provide connection paths to other such on-chip interconnect clusters. This arrangement can minimize hardware costs while allowing high-bandwidth connection paths within one cluster and lower aggregate bandwidth paths to all other clusters. For example, in one embodiment including a large number of cores, the cores can be grouped into multiple clusters, where each cluster includes eight processor cores (and corresponding ATE instances).

[0103] Figure 10 is a block diagram illustrating the use of a full crossbar interconnect matrix in a portion of a distributed computing system according to at least some embodiments. More specifically, Figure 10 is a block diagram illustrating the use of an interconnect matrix 1002 in a portion of the system 1000. In this example, the interconnect matrix 1002 connects the ATE master interfaces 0-n of n ATE instances within a single cluster (in Figure 10In some embodiments, each of the master interfaces 1004-1014 can be similar to Figure 2 The master interface 242 is shown, and each of the slave interfaces 1016-1026 can be similar to Figure 2 Slave interface 244 is shown.

[0104] In this example, the interconnect matrix 1002 can have full crossbar connectivity within this cluster of n ATE instances. In addition, if one of the n ATE instances connected by the interconnect matrix 1002 wants to access an ATE instance outside the cluster (e.g., send an RPC request frame or an RPC response frame to an ATE instance in another cluster of ATE instances), it can communicate with the remote ATE instance (or, more specifically, the interconnect matrix for the cluster to which the remote ATE instance is connected) through one of the interface ports (ISO-ISn) of the interconnect matrix 1002 (shown as the master interface 1050). More specifically, it can send the RPC request frame or the RPC response frame to a slave interface of the interconnect matrix for another such cluster through one of the master interfaces 1040. The slave interface of another interconnect matrix can be similar to Figure 10 1050 of the interconnect matrix 1002. Similarly, any of the n ATE instances in the cluster can receive an RPC request frame or an RPC response frame from a remote ATE instance (an ATE instance in a different cluster) via one of the slave interfaces 1040 of the interconnect matrix 1002. In other words, in this example, the ATE master interfaces M0-Mn can be connected to the slave ports of the interconnect matrix 1002, and the master ports of the interconnect matrix 1002 can be connected to the ATE slave ports S0-Sn. In general, the interconnect matrix 1002 can receive RPC frames from the master interface of a local ATE instance (one within the same cluster) and / or the master interface of a similar full crossbar interconnect of another cluster, and can send RPC frames to the slave interface of the local ATE interface (one within the same cluster) and / or the slave interface of a similar full crossbar interconnect of another cluster.

[0105] As shown in this example, the interconnect matrix 1002 can include multiple arbiters, such as arbiter 1030 and arbiter 1052. These arbiters can be used to select between multiple processor cores (or their ATE instances) that can be attempting to send RPC frames to another processor core (or its ATE instance) at any given time. For example, in a system that includes eight processor cores in each cluster, it can be possible that seven of these cores can be attempting to send something to the same (eighth) core. In some embodiments, it can even be possible that all eight cores are sending something to one core (e.g., if the ATE is attempting to send something to itself). In either of these cases, the arbiters can be configured to determine the order in which the processor cores are able to deliver their RPC frames to a given one of the ATE instances. In some embodiments, these arbiters can be configured to attempt to keep each given RPC frame from a particular core intact as it passes through the interconnect matrix 1002 (e.g., to prevent RPC frame mixing). By avoiding mixing and interaction between RPC frames, each RPC frame can be processed as a contiguous atomic block. Note that in some embodiments, it can be possible for an ATE to send something to itself without having to go through the interconnect. However, in other embodiments, it can be required for each ATE to send all RPCs through the interconnect even if the destination is the ATE itself.

[0106] Figure 11 is a block diagram illustrating an example connectivity for a four-cluster ATE system in accordance with at least some embodiments. In this example, each cluster has a full crossbar interconnect, such as the interconnect matrix 1002 shown in Figure 10 , and these interconnect matrices communicate with each other through master and slave interface ports to provide all-to-all connectivity. More specifically, the four-cluster ATE system 1100 includes four interconnected clusters 0-3 (shown as elements 1102, 1104, 1106, and 1008 in Figure 11 ), where each cluster includes an interconnect matrix. In this example, each interconnect matrix includes three inter-matrix master interfaces (IM0-IM2) and three inter-matrix slave interfaces (IS0-IS2) in addition to master and slave interfaces for exchanging messages between ATE instances within a cluster.

[0107] For example, bus 1101 can carry messages between the master and slave interfaces of interconnect cluster 0 (1102) and interconnect cluster 1 (1104) for the benefit of ATE instances within those clusters. Similarly, bus 1103 can carry messages between the master and slave interfaces of interconnect cluster 0 (1102) and interconnect cluster 3 (1108); bus 1105 can carry messages between the master and slave interfaces of interconnect cluster 1 (1104) and interconnect cluster 2 (1106); bus 1107 can carry messages between the master and slave interfaces of interconnect cluster 2 (1106) and interconnect cluster 3 (1108); bus 1109 can carry messages between the master and slave interfaces of interconnect cluster 0 (1102) and interconnect cluster 2 (1106); and bus 1110 can carry messages between the master and slave interfaces of interconnect cluster 1 (1104) and interconnect cluster 3 (1108).

[0108] Note that while in several examples ATE instances are described as working with (or including) receive queues that are FIFO queues, in other embodiments the receive queues can have different structures or ordering schemes. For example, in some embodiments ATE instances can maintain multiple queues for the RPCs they receive, and can place RPCs on individual receive queues based on the core from which they were received. In some embodiments, RPCs placed in one or more receive queues can be processed in an order based on the core from which they were received. In some such embodiments, ATE instances can give different priority to RPCs from different cores. In other embodiments, there can be multiple receive queues for each source processor or for a set of source processors, with each receive queue storing RPCs of similar priority. For example, there can be one receive queue for high-priority RPCs and another receive queue for low-priority RPCs. In other embodiments, other schemes for placing RPCs in different receive queues and / or for determining the order in which they will be processed can be implemented.

[0109] In general, the ATE subsystems described herein can allow shared data to remain in a local, low-access-latency data memory, rather than moving it to a high-latency (and possibly load-intensive) main system memory such as a DDR memory. Additionally, in some embodiments the ATE subsystems described herein can allow fine-grained control over the size of the shared region, and that size can be dynamically configured by software during runtime.

[0110] Example System

[0111] Techniques for utilizing an atomic transaction engine as described herein can be further illustrated by way of example with an example system that includes a multi-banked memory. In various embodiments, such a system can include a plurality of requesters (e.g., one or more general-purpose or special-purpose processor cores, one or more direct memory access (DMA) controllers, one or more ATE instances such as those described herein, network interface components, and / or peripheral devices), where each of the requesters can from time to time request access to one or more of the memory banks in order to perform a task. In various embodiments, the multi-banked memory can be configured as interleaved memory or non-interleaved memory as described herein, and a memory allocation component can be configured to arbitrate access to individual banks within the multi-banked memory. In one example, each of a plurality of subsystems within the system can include a low-power processor core that executes software for query processing. In this example, query processing can be a parallelizable task, and there can be multiple processor cores in the system, each executing the same code on independent partitions of data (e.g., data in portions of a distributed shared memory controlled by the processor cores).

[0112] Figure 12 is a block diagram illustrating a portion of a system that includes a shared multi-banked memory according to at least some embodiments, the system utilizing atomic transaction engine instances to manage and perform operations targeted to the shared multi-banked memory. In this example, among other things, this portion of the system (shown as subsystem 1200) includes four memory banks (shown as memory banks 1210a-1210d) that are shared among a plurality of requesters and can be collectively referred to as DMEM, a memory allocation / arbitration component 1220 (which can be a pipeline-type arbiter), and a memory interface 1230 (which can include and / or perform the functions of a data bus, a clock signal, a word address select, a chip select, a row / column select, and / or a bank select, among other things). In this example, potential requesters for the shared memory banks 1210 include a plurality of DMA components / controllers 1250, a general- or special-purpose core (shown as processor core 1240), an ATE instance 1270 associated with the processor core 1240, and one or more other requesters 1260 that can be similar or different component types. The ATE instance 1270 can implement a message passing interface (e.g., a message passing interface through which messages can pass between a processor core and a corresponding direct access memory or between two processor cores), as described herein.

[0113] In this example, each of the DMA components / controllers 1250 can be a DMA engine configured to pre-load a particular memory bank of the memory banks 1210a-d with data that the processor core 1240 wants to operate on. Once the data has been pre-loaded, the DMA engine can tell the processor core that it can proceed to operate on the data. When the processor core finishes operating on the data, it can tell the DMA engine, after which the DMS engine can be configured to offload the result(s) from the DMEM. In this example, the processor core 1240 can be a special purpose core (e.g., a database processor) that has special status within the system and / or higher priority than one or more of the other requesters (e.g., the DMA components / controllers 1250, the ATE instance 1270, and / or other requesters 1260) for requests for DMEM resources. Note that in some embodiments, the memory interface 1230 and / or the memory allocation / arbitration component 1220 can include one or more request buffers that can be configured to hold requests for DMEM resources until they have been processed and / or acted on (e.g., by granting or denying the request) by the memory allocation / arbitration component 1220. In some embodiments, the ATE instance 1270 can be associated with the processor core 1240, and the processor core 1240 can control a portion of the memory locations within the DMEM. As described herein, the ATE instance 1270 can communicate with other ATE instances (e.g., ATE instances in separate subsystems) via a dedicated low-latency ATE interconnect.

[0114] In this example, not all requesters can be able to direct requests to all memory banks 1210. For example, each of the DMA components / controllers 1250 can be associated with a respective one of the memory banks 1210a-1210d, and can not be potential requesters for any other of the memory banks 1210a-1210d. However, the memory allocation / arbitration component 1220 can be configured to direct requests from (or on behalf of) some or all of the processor cores 1240, the ATE instance 1270, and the other requesters 1260 to any given bank based on addresses specified in resource requests received from those sources. Thus, for each of the memory banks 1210a-1210d, there can be multiple requesters, including the processor cores 1240, one or more of the DMA components / controllers 1250 (each of which can be a requester for a separate one of the memory banks 1210a-1210d), the ATE instance 1270, and one or more of the other requesters 1260. Note that in some embodiments, the processor cores 1240 can be able to submit 32-bit requests that require access to a single one of the memory banks 1210a-1210d or 64-bit requests that require access to two of the memory banks 1210a-1210d.

[0115] In this example, each memory bank can be handled independently by the memory allocation / arbitration component 1220 from a logical standpoint, and each bank can be unaware of the other banks. Here, for requesters that can request multiple banks, the memory allocation / arbitration component can use addresses specified in each request to direct it to the appropriate memory bank. In some embodiments, the ATE instance 1270 can be configured to handle RPCs issued by the processor cores 1240 that are directed to the memory banks 1210 (or portions thereof) or RPCs received from another ATE instance (not shown) that represents a remote processor core that wishes to access the memory banks 1210 (or portions thereof).

[0116] Figure 13 A computing node configured to implement some or all of the methods described herein according to various embodiments is illustrated. The computing node 1300 can be any of various types of devices, including, but not limited to, a personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, handheld computer, workstation, network computer, a consumer device such as a cellular phone, an application server, a storage device, a peripheral device such as a switch, modem, router, or in general any type of computing device.

[0117] Some of the functionality as described herein for mechanisms utilizing an atomic transaction engine (or instance thereof) can be provided as a computer program product or software (or invoked by a computer program product or software), which can include a non-transitory computer readable storage medium having stored thereon instructions that can be used to program a computer system (or other electronic device) to perform a process according to various embodiments. The computer readable storage medium can include any mechanism for storing information in a form accessible by a machine (e.g., computer, etc.), such as software, processing application, etc. The computer readable storage medium can include, but is not limited to, magnetic storage medium (e.g., floppy diskettes, etc.), optical storage medium (e.g., CD-ROMs, etc.), magneto-optical storage medium, read-only memory (ROM), random-access memory (RAM), erasable programmable memory (e.g., EPROM & EEPROM, etc.), flash memory, electrical or other types of medium suitable for storing program instructions. Further, program instructions can be transmitted in a

[0118] In various embodiments, compute node 1300 can include one or more processors 1370; each processor 1370 can include multiple cores, any of which can be single-threaded or multi-threaded. For example, multiple processor cores can be included in a single processor chip (e.g., a single processor 1370), and multiple processor chips can be included on a CPU board, two or more of which can be included in compute node 1300. In various embodiments, each processor 1370 can include a cache hierarchy 1375. For example, each processor 1370 (or processor chip) can include one or more local caches (e.g., an LI cache per processor core and / or one or more caches shared by multiple processor cores on a single processor chip, such as an L2 cache and / or an L3 cache). As shown in this example, compute node 1300 can include a memory controller 1348 and / or a network interface 1346. Compute node 1300 can also include one or more shared resources 1342, one or more ATE instances 1355 (e.g., one per processor core), one or more other requesters 1344, and a shared resource arbiter 1380 (which, in different embodiments, can be a pipeline-type arbiter or a non-pipeline-type arbiter). In various embodiments, one or more of processors 1370 can be (or include) a special-purpose processor core, such as a database core described herein, and any or all of processors 1370 can be requesters for various shared resources 1342 of compute node 1300 and / or for shared memory location 1335. In some embodiments, shared memory location 1335 can include a multi-bank memory such as the DMEM described above, any one or more of processors 1370, ATE instances 1355, and / or other requesters 1344 can request access to the multi-bank memory. ATE 1355 can be configured to implement some or all of the techniques described herein for controlling and performing accesses to shared memory location 1335 received from other ATE instances 1355 and / or processors 1370. Arbiter 1380 can be configured to control accesses to other shared resources 1342 received from multiple of ATE instances 1355 or other requesters 1344 and / or processors 1370.

[0119] As Figure 13As shown in FIG. 13, computing node 1300 can also include or have access to one or more persistent storage devices 1350 (e.g., optical storage, magnetic storage, hard disk drive, tape drive, solid state memory, etc. external to computing node 1300 but accessible by computing node 1300) and one or more system memories 1310 (e.g., cache, SRAM, DRAM, RDRAM, EDO RAM, DDR 10 RAM, SDRAM, Rambus RAM, EEPROM, etc.). In some embodiments, system memory 1310 can store a portion of a dataset for an application that processor(s) 1370 can operate on locally (e.g., in some cases by using RPCs generated as described herein). For example, memory 1310 on computing node 1300 can include local storage for a data structure (e.g., within private memory location 1330 or shared memory location 1335) whose elements can be the target of various functions of an application. In some embodiments, memory 1310 can include persistent storage (e.g., for storing firmware or configuration parameter values that are fixed for a given computing node or a given set of configuration registers). In other embodiments, configuration registers or configuration parameters stored in system memory 1300 can include Figure 13 Fewer or additional components (e.g., video cards, audio cards, additional network interfaces, peripherals, network interfaces such as ATM interfaces, Ethernet interfaces, frame relay interfaces, etc.) not shown in FIG. 13.

[0120] In this example, one or more processors 1370, memory controller 1348 (which can provide access to storage 1350), shared resources 1342, ATE instance 1355, other requesters 1344, arbiter 1380, network interface 1346, and system memory 1310 can be coupled to system interconnect 1340. One or more of system memory 1310 can contain program instructions 1320. Program instructions 1320 can be executable to implement one or more applications 1322 (which can include all or a portion of an application that accesses shared memory in order to perform its functions), shared libraries 1324, or operating system 1326. Program instructions 1320 can be encoded in platform native binary, in any interpreted language (such as Java byte code), or in any other language (such as C / C++, Java TM byte code), or in any other language (such as C / C++, Java TMThe program instructions 1320 can include functionality, operations, and / or other processes as described herein for performing various RPCs or initiating performance of various RPCs. In various embodiments, such support and functionality can be present in one or more of the shared libraries 1324, operating system 1326, or applications 1322. The system memory 1310 can further include private memory locations 1330 and / or shared memory locations 1335 in which data (including portions of data sets for the applications 1322 and / or various configuration parameter values) can be stored. For example, in various embodiments, the private memory locations 1330 and / or shared memory locations 1335 can store data accessible to simultaneously executing threads, processes, or transactions. Additionally, any of the system memory 1310 and / or caches of the processor(s) 1370 can at various times store data that has been fetched (or pre-fetched) from one of the system memory 1310 and / or from the storage 1350 by an application executing on the compute node 1300 (or on behalf of an application executing on the compute node 1300). Note that any of the system memory 1310 and / or caches of the processor(s) 1370 can at various times store any other data useful in implementing the technology described herein, some of which can include values that can be configured by a programmer or user.

[0121] While the above embodiments have been described in considerable detail, numerous variations and modifications will now be apparent to those skilled in the art once the above disclosure is fully appreciated. For example, while many embodiments are described in terms of particular types of memory, processor cores, and / or requesters, it should be noted that the technology and mechanisms disclosed herein for utilizing an atomic transaction engine (or instance thereof) can be applicable in other contexts in which multiple processors operate on shared memory. The appended claims are intended to be construed as embracing all such variations and modifications.

Claims

1. A method for a distributed shared memory system, comprising: Executing instructions on a processor core of a plurality of processor cores, wherein each processor core is coupled to an atomic transaction engine instance and a memory arbitration component distinct from the atomic transaction engine instance via a memory interface, the executing comprising: identifying that the instruction comprises an access operation targeting a location in the distributed shared memory; In response to the identifying, sending parameters for the access operation to the corresponding atomic transaction engine instance via the corresponding memory interface; Determining, by the atomic transaction engine instance based on the received parameters, that the targeted location in the distributed shared memory is controlled by another atomic transaction engine instance, and in response to the determination: Sending, by the atomic transaction engine instance, a request to execute the access operation to the other atomic transaction engine instance via the interconnection; The access operation is executed through the other atomic transaction engine instance, or the execution of the access operation by another processor core among the multiple processor cores is initiated through the other atomic transaction engine instance, wherein the execution includes accessing the memory via the corresponding memory arbitration component of the other processor core.

2. The method according to claim 1, wherein The determining further comprises: executing additional instructions on said processor core; or The targeted location of the distributed shared memory is determined by pipeline circuitry within the processor core and is controlled by another atomic transaction engine instance.

3. The method according to claim 1 or 2, further comprising: Receiving response data to the request from the other atomic transaction engine instance through the atomic transaction engine instance; And in response to receiving: Returning the response data to the processor core through the atomic transaction engine instance; or The response data is written, by the atomic transaction engine instance, to a location in memory from which the processor core expects to retrieve the response data.

4. The method according to claim 1 or 2, in, The access operations target a plurality of portions of a distributed shared memory, each portion being controlled by an atomic transaction engine instance coupled to a different one of the plurality of processor cores; wherein sending the request to perform the access operation comprises sending the request to a corresponding atomic transaction engine instance coupled to a different processor core among the plurality of processor cores; and Executing the access operation includes executing the access operation or initiating execution of the access operation by a corresponding atomic transaction engine instance coupled to a different processor core among the multiple processor cores.

5. The method according to claim 1 or 2, in, Executing the access operation through the other atomic transaction engine instance, or initiating execution of the access operation by the other processor core through the other atomic transaction engine instance includes: determining whether the access operation can be performed by circuitry within the another atomic transaction engine instance without intervention by the another processor core; In response to determining that the access operation can be performed by circuitry within the another atomic transaction engine instance without intervention by the another processor core, performing the access operation by the another atomic transaction engine instance; and In response to determining that the access operation cannot be executed by circuitry within the other atomic transaction engine instance without intervention of the other processor core, initiating execution of the access operation by another processor core among the multiple processor cores through the other atomic transaction engine instance.

6. The method according to claim 1 or 2, in, Initiating, by the another atomic transaction engine instance, another processor core among the plurality of processor cores to execute the access operation includes: writing information about the access operation to one or more memory locations accessible to the other processor core; and An interrupt is issued to the other processor core indicating that the access operation should be performed by the other processor core.

7. A distributed shared memory system comprising: a plurality of processor cores, each processor core coupled via a memory interface to an atomic transaction engine instance and a memory arbitration component distinct from the atomic transaction engine instance; The processor cores of the plurality of processor cores are configured to execute instructions of a distributed application, wherein, in order to execute the instructions, the processor core is configured to: identifying that the instruction comprises an access operation targeting a location in the distributed shared memory; and In response to the identifying, sending parameters for the access operation to the corresponding atomic transaction engine instance via the corresponding memory interface; The atomic transaction engine instance is configured as follows: Based on the received parameters, determining that the targeted location in the distributed shared memory is controlled by another atomic transaction engine instance; and In response to the determination, sending a request to perform the access operation to the other atomic transaction engine instance via the interconnect; and Wherein, the another atomic transaction engine instance is configured as follows: receiving a request to perform the access operation from the atomic transaction engine instance via the interconnect; and The access operation is executed or execution of the access operation by another processor core of the plurality of processor cores is initiated, the execution including accessing a memory via a corresponding memory arbitration component of the another processor core.

8. The distributed shared memory system according to claim 7, wherein: During execution of the access operation, the other atomic transaction engine instance is configured to collect response data associated with execution of the identified operation.

9. The distributed shared memory system according to claim 7 or 8, in, The another atomic transaction engine instance includes circuitry configured to perform access operations of a plurality of operation types; and In which, in order to determine whether the access operation can be executed by the circuit within the other atomic transaction engine instance without the intervention of the other processor core, the other atomic transaction engine instance is configured to determine whether the type of the access operation is one of the multiple operation types that can be executed by the circuit within the other atomic transaction engine instance.

10. The distributed shared memory system according to claim 9, wherein: The access operation includes a series of operations identified by information in the received request, wherein each operation in the series of operations is a type of operation that can be performed by circuitry within the other atomic transaction engine instance.

11. The distributed shared memory system according to claim 7 or 8, in, The another atomic transaction engine instance is configured to, in response to determining that the access operation cannot be performed by circuitry within the another atomic transaction engine instance without intervention by the another processor core, initiate performance of the access operation by the another processor core; and In order to initiate execution of the access operation by the other processor core, the other atomic transaction engine instance is configured as follows: writing information about the access operation to one or more memory locations accessible to the other processor core; as well as An interrupt is issued to the other processor core indicating that the access operation should be performed by the other processor core.

12. The distributed shared memory system according to claim 11, wherein: In response to the interrupt, the another processor core is configured to perform the access operation.

13. The distributed shared memory system according to claim 7 or 8, wherein: The another atomic transaction engine instance is further configured as follows: generating a response associated with performance of the access operation; and Return the response to the atomic transaction engine instance.

14. A distributed shared memory system comprising: a plurality of processor cores, each processor core coupled via a memory interface to an atomic transaction engine instance and a memory arbitration component distinct from the atomic transaction engine instance; as well as a distributed shared memory, wherein each processor core of the plurality of processor cores controls a respective portion of the distributed shared memory; The atomic transaction engine instance coupled to a processor core of the plurality of processor cores includes circuitry configured to perform the following operations: retrieving a request from a receive queue local to the atomic transaction engine instance, wherein the request identifies an operation targeted at a location in the distributed shared memory controlled by a processor core coupled to the atomic transaction engine instance; The identified operation is executed, the execution including accessing a memory via a corresponding memory arbitration component coupled to a processor core of the atomic transaction engine instance.

15. The distributed shared memory system according to claim 14, wherein: The request is generated by another atomic transaction engine instance on behalf of a processor core among the multiple processor cores coupled to the other atomic transaction engine instance, and is transmitted to the atomic transaction engine instance by the other atomic transaction engine instance.

16. The distributed shared memory system according to claim 14 or 15, wherein: The request is generated by the atomic transaction engine instance on behalf of a processor core coupled to the atomic transaction engine instance and is placed in the receive queue by the atomic transaction engine instance.

17. The distributed shared memory system according to claim 14, in, The receiving queue is one of a plurality of receiving queues local to the atomic transaction engine instance; and Each of the multiple receiving queues stores requests received from different atomic transaction engine instances among the multiple atomic transaction engine instances, or stores requests including operations of different types or with different priorities.

18. The distributed shared memory system according to claim 14, in, Atomic transaction engine instances coupled to respective ones of the plurality of processor cores communicate with each other via a dedicated low-latency interconnect.

19. The distributed shared memory system according to claim 14, in, The system includes two or more clusters of processor cores, each cluster including a plurality of processor cores and a corresponding crossbar switch, respective atomic transaction instances associated with each of the plurality of processor cores communicating with each other via the crossbar switch; and The corresponding atomic transaction instances coupled to the processor cores in each cluster communicate with the atomic transaction instances coupled to the processor cores in other clusters via the interconnection between the corresponding crossbar switches.

20. The distributed shared memory system according to claim 14, in, a processor core coupled to the atomic transaction engine instance comprising address decoding circuitry configured to determine whether an operation targets a location in the distributed shared memory; and Wherein, the processor core coupled to the atomic transaction engine instance is configured as follows: During execution of a distributed application, determining that an operation of the distributed application targets a location in the distributed shared memory; and In response to determining that the operation of the distributed application targets a location in the distributed shared memory: refraining from advancing the operation within pipeline circuitry of a processor core coupled to the atomic transaction engine instance; and Information about the operation of the distributed application is provided to the atomic transaction engine instance that can be used to process the operation of the distributed application.

21. A method for a distributed shared memory system, comprising: receiving, at a local atomic transaction engine, information describing an atomic transaction to be performed at a memory address of a distributed shared memory, and in response to the receiving: In response to determining that the memory address is controlled by the local atomic transaction engine instance, causing the atomic transaction to be performed on the memory address; as well as In response to determining that the memory address is not controlled by the local atomic transaction engine instance, information describing the atomic transaction to be performed at the memory address is sent to a remote atomic transaction engine instance that controls the memory address.

22. The method according to claim 21, wherein Information describing the atomic transaction to be performed at the memory address is received from the processor in response to determining that an instruction to be executed by the processor coupled to the local atomic transaction engine instance accesses the distributed shared memory.

23. The method according to claim 21 or 22, wherein Information describing the atomic transaction to be performed at the memory address is received from another atomic transaction engine instance that is different from the local atomic transaction engine instance.

24. The method according to claim 21 or 22, in, The atomic transaction targets a plurality of addresses of the distributed shared memory that are respectively controlled by different atomic transaction engine instances; Wherein, sending information describing the atomic transaction to be executed includes sending information including corresponding addresses to different atomic transaction engine instances.

25. The method according to claim 22, wherein Causing the atomic transaction to be performed on the memory address includes: In response to determining that the atomic transaction can be executed by circuitry within the local atomic transaction engine instance without intervention of the processor, executing the atomic transaction by the local atomic transaction engine instance; and In response to determining that the atomic transaction cannot be executed by circuitry within the local atomic transaction engine instance without intervention by the processor, initiating execution of the atomic transaction by the processor through the local atomic transaction engine instance.

26. The method according to claim 25, wherein Initiating the execution of the atomic transaction by the processor through the local atomic transaction engine instance includes: writing information describing the atomic transaction to one or more memory locations accessible to the processor; and An interrupt is issued to the processor indicating that the atomic transaction should be executed by the processor.

27. The method according to claim 21 or 22, wherein The local atomic transaction engine instance and the remote atomic transaction engine instance communicate with each other via a dedicated low-latency interconnect.

28. An apparatus for a distributed shared memory system, comprising: A local atomic transaction engine instance coupled to a processor and a memory, wherein the memory implements a portion of a distributed shared memory, and wherein the local atomic transaction engine instance is configured to: receiving information describing an atomic transaction to be performed at a memory address of the distributed shared memory, and in response to the receiving: In response to determining that the memory address is controlled by the local atomic transaction engine instance, causing the atomic transaction to be performed on the memory address; and In response to determining that the memory address is not controlled by the local atomic transaction engine instance, information describing the atomic transaction to be performed at the memory address is sent to a remote atomic transaction engine instance that controls the memory address.

29. The apparatus according to claim 28, wherein Information describing the atomic transaction to be performed at the memory address is received from the processor in response to determining that an instruction to be executed by the processor coupled to the local atomic transaction engine instance accesses the distributed shared memory.

30. The device according to claim 28 or 29, wherein Information describing the atomic transaction to be performed at the memory address is received from another atomic transaction engine instance that is different from the local atomic transaction engine instance.

31. The device according to claim 28 or 29, in, The atomic transaction targets a plurality of addresses of the distributed shared memory that are respectively controlled by different atomic transaction engine instances; Wherein, sending information describing the atomic transaction to be executed includes sending information including corresponding addresses to different atomic transaction engine instances.

32. The apparatus according to claim 28 or 29, wherein Causing the atomic transaction to be performed on the memory address includes: In response to determining that the atomic transaction can be executed by circuitry within the local atomic transaction engine instance without intervention of the processor, executing the atomic transaction by the local atomic transaction engine instance; and In response to determining that the atomic transaction cannot be executed by circuitry within the local atomic transaction engine instance without intervention by the processor, initiating execution of the atomic transaction by the processor through the local atomic transaction engine instance.

33. The apparatus according to claim 32, wherein Initiating the execution of the atomic transaction by the processor through the local atomic transaction engine instance includes: writing information describing the atomic transaction to one or more memory locations accessible to the processor; and An interrupt is issued to the processor indicating that the atomic transaction should be executed by the processor.

34. A distributed shared memory system comprising: A plurality of atomic transaction engine instances are respectively coupled to corresponding processors and corresponding memories, wherein the corresponding memories together implement a distributed shared memory, and wherein a local atomic transaction engine instance among the plurality of atomic transaction engine instances is configured as follows: receiving information describing an atomic transaction to be performed at a memory address of the distributed shared memory, and in response to the receiving: In response to determining that the memory address is controlled by the local atomic transaction engine instance, causing the atomic transaction to be performed on the memory address; and In response to determining that the memory address is not controlled by the local atomic transaction engine instance, information describing the atomic transaction to be performed at the memory address is sent to a remote atomic transaction engine instance among the multiple atomic transaction engine instances that controls the memory address.

35. The distributed shared memory system according to claim 34, wherein: Information describing the atomic transaction to be performed at the memory address is received from the processor in response to determining that an instruction to be executed by the processor coupled to the local atomic transaction engine instance accesses the distributed shared memory.

36. The distributed shared memory system according to claim 34 or 35, wherein: Information describing the atomic transaction to be performed at the memory address is received from another atomic transaction engine instance that is different from the local atomic transaction engine instance.

37. The distributed shared memory system according to claim 34 or 35, in, The atomic transaction targets a plurality of addresses of the distributed shared memory that are respectively controlled by different atomic transaction engine instances; Wherein, sending information describing the atomic transaction to be executed includes sending information including corresponding addresses to different atomic transaction engine instances.

38. The distributed shared memory system according to claim 34 or 35, wherein: Causing the atomic transaction to be performed on the memory address includes: In response to determining that the atomic transaction can be executed by circuitry within the local atomic transaction engine instance without intervention of the processor, executing the atomic transaction by the local atomic transaction engine instance; and In response to determining that the atomic transaction cannot be executed by circuitry within the local atomic transaction engine instance without intervention by the processor, initiating execution of the atomic transaction by the processor through the local atomic transaction engine instance.

39. The distributed shared memory system according to claim 38, wherein: Initiating the execution of the atomic transaction by the processor through the local atomic transaction engine instance includes: writing information describing the atomic transaction to one or more memory locations accessible to the processor; and An interrupt is issued to the processor indicating that the atomic transaction should be executed by the processor.

40. The distributed shared memory system according to claim 34 or 35, wherein: The local atomic transaction engine instance and the remote atomic transaction engine instance communicate with each other via a dedicated low-latency interconnect.

Images