Two-level context caching and eviction for scattered aggregated DMA

Abstract

The invention provides a two-level context caching and eviction for distributed aggregation DMA (direct memory access). One aspect of the present disclosure may provide a system and method for processing a decentralized aggregated direct memory access (S-G DMA) instruction. During operation, the system may receive an S-G DMA instruction associated with a message and aggregate an instruction context of the S-G DMA instruction. The S-G DMA processor may process the S-G DMA instruction based on the aggregated instruction context and determine whether there is a pending S-G DMA instruction associated with the message. In response to the presence of the pending S-G DMA instruction, the system stores an instruction context at an address in the thermal context cache corresponding to the pending S-G DMA instruction. In response to the absence of the pending S-G DMA instruction, the system stores an instruction context in a cold context cache.

Description

[0001] Government-funded research statement

[0002] This invention was carried out with government support under contract number H98230-23-C-0350 granted by the Maryland Procurement Office. The government holds certain rights to this invention. background Technical Field

[0003] This disclosure generally relates to distributed clustered direct memory access (DMA). More specifically, this disclosure relates to context caching in distributed clustered DMA operations. Background Technology

[0004] Distributed-clustered direct memory access (DMA) is a sophisticated technique employed in applications requiring high-bandwidth and low-latency data transfer between memory and peripheral devices, such as high-performance computing (HPC) systems. This approach offers significant advantages over traditional DMA methods. Unlike traditional DMA, which is limited to transferring data in single contiguous blocks, distributed-clustered DMA allows for the transfer of non-contiguous memory blocks, providing greater flexibility in memory usage and potentially resulting in substantial performance improvements.

[0005] The Message Passing Interface (MPI) allows parallel tasks in HPC applications to collaborate through message exchange. When using distributed aggregated DMA, messages can be transmitted as a series of packets, and each packet transmission typically involves processing a corresponding instruction containing critical information such as memory address and size.

[0006] After processing an instruction, it is beneficial to cache the instruction's context (e.g., the memory address of the DMA operation associated with the instruction) for use by subsequent instructions, thereby reducing DMA processing time and bandwidth usage. Without context caching, a starting context would need to be built for each instruction, requiring lengthy DMA reads from host memory. Host memory bandwidth is wasted because the same context is read from memory multiple times. Context caching can also be useful when instruction processing is temporarily paused. Attached Figure Description

[0007] Figure 1 The illustration shows an example schematic diagram of a distributed clustered direct memory access (SG) DMA engine according to one aspect of this application.

[0008] Figure 2 A flowchart illustrating an example process for processing distributed aggregated DMA instructions and caching instruction contexts according to one aspect of this application is presented.

[0009] Figure 3A A flowchart illustrating an example process for processing various context cache commands for reading the hot context cache, according to one aspect of this application, is presented.

[0010] Figure 3B A flowchart illustrating an example process for transferring context from a hot context cache to a cold context cache, according to one aspect of this application, is presented.

[0011] Figure 3C A flowchart illustrating an example process for processing various context cache commands for reading cold context caches, according to one aspect of this application, is presented.

[0012] Figure 4 An example network interface controller (NIC) according to one aspect of this application is illustrated.

[0013] Figure 5 The illustration shows a computer-readable medium facilitating the operation of a two-level caching system according to one aspect of this application.

[0014] In these accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0015] In the context of MPI, distributed-aggregate DMA (SG DMA) offers several advantages, including high memory utilization and reduced CPU overhead. For example, MPI allows users to provide the base address to start a transfer and the data type applied to that transfer (which describes the layout of the data in memory). MPI also provides a mechanism called "derived data types," which can describe more complex data structures and use that description for certain MPI calls (e.g., MPI_Send and MPI_Recv). Historically, using derived data types in MPI can have resulted in poor performance because the library must interpret the data description to perform copying to and from the internal buffer for the MPI_Send and MPI_Recv commands, respectively. Therefore, packing and unpacking operations are still performed in software and are still serial with communication, which can lead to overall performance inefficiency for data-related operations such as distributed-aggregate DMA operations.

[0016] Hardware solutions have been developed to improve data manipulation efficiency. For example, a network interface card (NIC) can incorporate a specially designed distributed-aggregate (SG) DMA engine (also known as a data type engine). This engine allows users to describe complex data layouts for sending and receiving data, including multidimensional arrays with unique strides in each of multiple dimensions. The SG DMA engine can perform the aforementioned distributed-aggregate DMA operations, eliminating the inefficient data copying used in previous software solutions. The SG DMA engine can reside in the network interface card (NIC) or its application-specific integrated circuit (ASIC), including as part of both outbound and inbound packet engines.

[0017] The SG DMA engine can further reduce DMA processing time and host memory bandwidth consumption by caching instruction context. While context caching offers significant benefits, efficient cache management can be challenging.

[0018] In some aspects of this disclosure, to improve the efficiency of instruction context storage and transfer, the SG DMA engine on the host's network interface controller (NIC) may include a two-level context cache system for caching instruction context in both paused and completed processing scenarios. This two-level cache system may include a hot context cache that can be directly written to by the SG DMA processor, and a cold context cache that can be written to without direct SG DMA processor input. The hot context cache can be used to store urgently needed instruction context (i.e., hot context), while the cold context cache can be used to store instruction context that may not be needed for a period of time (i.e., cold context). In some aspects, the hot context cache may include a single-way cache, while the cold context cache may include a four-way cache or a single-way cache.

[0019] In a completed scenario, after processing an instruction, the SG DMA engine can determine if there are any pending instructions for the same message (i.e., whether the instruction exists in the SG DMA engine's instruction tracker or buffer). If so, the SG DMA engine can cache the instruction context in the hot context cache at the address corresponding to the pending instruction. This allows the instruction context to be transferred directly from the preceding instruction (i.e., the processed instruction) to the following instruction (i.e., the pending instruction) without reloading from host memory. If no pending instructions exist, the instruction context can be stored in a cold context cache, which can be accessed by instructions for the same message when needed. After all instructions for the message have been processed (e.g., after processing instructions marked as message end), entries related to that message are evicted from either the hot or cold context cache.

[0020] In paused scenarios, the context of the paused instruction can be written to the hot context cache at the address corresponding to the paused instruction. When the SG DMA processor resumes processing the instruction, it can read the instruction context from the hot context cache.

[0021] Figure 1 The illustration shows an example schematic of a distributed aggregated direct memory access (SG) DMA engine according to one aspect of this application. The SG DMA engine can be part of a network interface card (NIC), such as an application-specific integrated circuit (ASIC) of the NIC. More specifically, the NIC can include SG DMA engines on both the transmit and receive paths to process SG DMA instructions associated with transmitted and received messages.

[0022] exist Figure 1 In the example shown, the SG DMA engine 100 includes an instruction tracker 102, an arbitrator 104, an SG DMA pipeline 106, an SG DMA processor 108, a hot context cache 110, a cold context cache 112, and an eviction queue 114. The SG DMA engine 100 may include... Figure 1 The components shown are more numerous. For example, various logical units for managing caches (including hot context cache 110 and cold context cache 112) are included. Figure 1 Not shown in the diagram. The SG DMA engine 100 may also include buffers storing Input / Output Vector (IOVEC) entries and / or context fast-forward entries associated with instruction contexts cached in hot context cache 110 and cold context cache 112. Each IVOEC entry may include a list of addresses (or offsets) and lengths. Each context fast-forward entry may include a description of the starting point (e.g., a loop counter value and byte offset within a data element) within a nested loop memory access pattern. This buffer is referred to as an ICB buffer. The instruction context containing the list of ICB entries may be referred to as an IVOEC processing context, while the instruction context containing the nested loop counter value may be referred to as a derived data type processing context. The SG DMA engine 100 may further include a descriptor storage device storing data type descriptors. Each data type descriptor may contain various information about the SG DMA operation to be performed. In one example, the data type descriptor may represent a multidimensional array defined by the number of data elements per dimension, the data element size, and the stride. In another example, the data type descriptor may reference an IVOEC structure with absolute or relative addresses.

[0023] Instruction tracker 102 may include a memory block that buffers multiple pending data type (or SG DMA) instructions associated with multiple packets. Note that pending instructions in tracker 102 may be associated with different messages. In some aspects, data type instructions corresponding to packets of a message may enter SG DMA engine 100 sequentially (i.e., in the order in which the packets are constructed and transmitted by the NIC). Alternatively, in the case where the NIC receives packets out of order, data type instructions corresponding to packets of a message may enter SG DMA engine 100 out of order. In some examples, instruction tracker 102 may include 256 entries. Each data type instruction may include a message ID and a sequence number (e.g., a packet sequence number). The message ID can be used to associate tracked instructions for the same message. Different high-level identifiers for different types of messages (e.g., Put requests vs. Get responses; unrestricted vs. restricted) can be encoded into a common message ID to allow for uniform context caching for different types of messages.

[0024] Arbitrator 104 can be responsible for selecting instructions from instruction tracker 102 to send to SG DMA pipeline 106. In some respects, arbitrator 104 can perform round-robin arbitration among entries in tracker 102. In addition to entries in tracker 102 (e.g., SG DMA instructions), commands for deallocation messages (referred to as DEALLOC_M commands) can also gain access to SG DMA pipeline 106. Some tracker entries may attempt to transfer instruction context from hot context buffer 110 to cold context buffer 112 (e.g., HOT_2COLD commands), and such entries have the highest priority when accessing SG DMA pipeline 106, followed by DEALLOC_M commands. Tracker entries ready to perform any processing other than hot-to-cold buffer transfers have the lowest priority and can be arbitrated in a round-robin manner.

[0025] The SG DMA pipeline 106 is responsible for aggregating the information required to process data type instructions. For example, for an instruction entering the SG DMA pipeline 106 (i.e., after winning arbitration), the pipeline can determine whether the instruction context (e.g., the number of bytes transferred so far, information about ICB buffer entries, etc.) can be found in the cache (hot context cache 110 or cold context cache 112). If it can be found, the SG DMA pipeline 106 can obtain the cached context. Otherwise, the SG DMA pipeline 106 can obtain the context from host memory using the data type descriptor associated with the instruction.

[0026] The instruction context may include a pointer to an ICB buffer, and the SG DMA pipeline 106 may obtain the corresponding ICB entry (e.g., one or more IVOEC entries or data type fast-forward entries). Given a three-dimensional array of data elements in host memory, the data type fast-forward entry may include multiple elements in each dimension, the stride in each dimension, and the size of the data element to be transferred. On the other hand, an IVOEC entry may include a list of addresses (or offsets when relative addressing is used) and lengths.

[0027] The SG DMA processor 108 is responsible for processing data types or SG DMA instructions based on the instruction context aggregated by the SG DMA pipeline 106. In some aspects, for each data type or SG DMA instruction, the SG DMA processor 108 can generate one or more DMA instructions that define multiple DMA read / write operations. For example, in the outbound direction, the SG DMA processor 108 can issue a DMA read instruction, which results in a read request to host memory using address information identified by the instruction context. In the inbound direction, the SG DMA processor 108 can issue a DMA write instruction, which results in the transfer of the payload in a received packet to host memory.

[0028] When the SG DMA processor 108 suspends instruction processing (e.g., due to depleted output queue credits), the SG DMA pipeline 106 can write the current instruction context to the hot context cache 110, and the corresponding tracker entry in the instruction tracker 102 can request arbitration to immediately gain access to the SG DMA pipeline 106.

[0029] After the SG DMA processor 108 has finished processing an instruction (e.g., after the entire payload of the corresponding packet has been transferred to or from host memory), the processor can determine whether there are any pending instructions for the same message. In some aspects, the SG DMA processor 108 can determine whether there is an instruction with the same message ID and subsequent packet sequence number in the instruction tracker 102. If so, the SG DMA processor 108 can instruct the SG DMA pipeline 106 to write the instruction context into the hot context cache 110. More specifically, the instruction context can be written into the hot context cache 110 at the position corresponding to the next pending instruction for the same message. In some aspects, the entry number in the hot context cache 110 corresponds to the entry number in the instruction tracker 102, and the tracker entry number can be used as a location index (e.g., a cache index) for the cache context within the hot context cache 110. For example, if the tracker entry number for the next pending instruction is i, the context of the processed instruction can be written into the hot context cache 110 as the i-th entry. When a pending instruction enters the SG DMA pipeline 106, the corresponding instruction context can be retrieved from the hot context cache 110 using index i. In this way, the instruction context of the preceding instruction can be directly transferred to the following instruction.

[0030] If no pending instructions with the same message are found (i.e., no instruction with the same message ID and subsequent sequence number is found in instruction tracker 102), SG DMA processor 108 can instruct SG DMA pipeline 106 to transfer the instruction context to the address in cold context cache 110 (e.g., using the HOT_2COLD command) obtained based on the message ID and the increment of the group sequence number included in the instruction. More specifically, because SG DMA processor 108 does not write directly to the cold context cache, the instruction context can first be written back to the location in hot context cache 110 corresponding to the processed instruction, and then transferred to cold context cache 112. In some respects, after the context is written to hot context cache 110, the entry in instruction tracker 102 that has just completed processing of the instruction can be placed in the HOT_2COLD state. The entry in the HOT_2COLD state will arbitrate access to SG DMA pipeline 106. When the entry accesses the pipeline, a HOT_2COLD command can be generated and submitted to hot context cache 110. As previously mentioned, the tracer entry that generates the HOT_2COLD command has the highest priority when accessing the SG DMA pipeline 106, thereby ensuring the timely transfer of instruction context.

[0031] In some respects, the message ID can be used, in whole or in part, as an address for the cache context within the cold context cache 112 (e.g., the address of each cache path in random access memory (RAM)). In this respect, the message ID and the packet sequence number can be used as keys to fully identify the cache context. When the SG DMA pipeline 106 receives an instruction that may belong to the same message as a previously processed instruction, the pipeline can use the message ID and the packet sequence number as search keys to search the cold context cache 112. In one example, the cold cache 112 may be a four-way cache, and the SG DMA pipeline 106 can read the entry at the address corresponding to the message ID in each path of the cold cache 112 and check the search key in each valid entry. If a matching entry is found (i.e., it has the same message ID and packet sequence number), the context can be retrieved from the cold context cache 112, and the cache entry is subsequently invalidated. Otherwise, the cold cache 112 can indicate that no matching instruction context was found.

[0032] Eviction queue 114 is responsible for queuing contexts from evicted context cache entries. The context from a context cache entry may contain a list of ICB entries that need to be released when the context is evicted. In one example, the last instruction of a message may be marked as message end. After processing the message end instruction, SG DMA pipeline 106 may transfer the instruction context corresponding to the message end instruction from hot context cache 110 and a list of any possible ICB entries referenced in the context to eviction queue 114. Additionally, when SG DMA engine 100 receives a DEALLOC_M command (indicating that a message with the corresponding message ID has been fully processed), the engine may read cold context cache 112 (e.g., using the message ID as an address) to locate and transfer any entries with matching message IDs to eviction queue 114. After eviction, these cold cache entries will be invalidated.

[0033] Figure 2 A flowchart illustrating an example process for processing distributed aggregated DMA instructions and caching instruction context, according to one aspect of this application, is presented. This process can be executed by an SG DMA engine on the host device's NIC. The SG DMA engine can be implemented using hardware, firmware, software components, or a combination thereof.

[0034] During operation, the SG DMA pipeline (which may be part of the SG DMA engine) may receive SG DMA instructions associated with a message (Operation 202). In some aspects, the message may be an MPI message, and the SG DMA instruction may correspond to a packet belonging to the message (which may be a packet to be transmitted or already received). The SG DMA instruction may include a message ID and a packet sequence number. In some aspects, the SG DMA instruction may be selected from the instruction tracker by an arbitrator. In a further aspect, the arbitrator may use a polling arbitration scheme to select a tracker entry to access the SG DMA engine.

[0035] The SG DMA pipeline in the SG DMA engine can aggregate the instruction context of SG DMA instructions (Operation 204). The instruction context can reflect the state of the SG DMA engine (e.g., hardware and / or software state), including the state of tracker entries and the state of the SG DMA processor. More specifically, as an instruction traverses the SG DMA pipeline, the pipeline can collect information such as the number of bytes transferred for that instruction so far, one or more pointers to the ICB buffer. Note that ICB entries can include IVOEC entries and / or data type fast-forward entries. If a set of IVOEC entries is used to describe a host memory location to be accessed, the instruction context can include a pointer to the first IVOEC entry in that set. If data type fast-forward entries are used to describe these memory locations, the instruction context can include the cycle count for each dimension of the three-dimensional memory array and the byte offset within the starting data element.

[0036] Then, the SG DMA processor in the SG DMA engine can process S-GDMA instructions based on the aggregated instruction context (operation 206). In some aspects, the SG DMA processor can generate one or more DMA commands based on address and size information included in the ICB entry referenced by the instruction context. The generated DMA commands (e.g., DMA read or write commands) can load data from or write data to the corresponding memory location.

[0037] After processing the SG DMA instruction, the engine can determine whether there are any pending S-GDMA instructions associated with the same message (Operation 208). In some respects, the instruction tracker can be searched to determine whether there are any instructions with the same message ID in the tracker.

[0038] In response to the existence of a pending SG DMA instruction, the processor can send the instruction context to the address in the hot context cache corresponding to the pending SG DMA instruction (operation 210). More specifically, the index of the hot context cache entry (e.g., its entry number) can correspond to the tracker entry number of the pending SG DMA instruction. The hot context cache can be directly accessed by the SG DMA processor, meaning that the SG DMA processor can directly write to the hot context cache.

[0039] Subsequently, the SG DMA processor can process pending SG DMA instructions associated with the same message (operation 212) by retrieving the instruction context from the hot context cache.

[0040] In response to the absence of any pending SG DMA instructions, the processor can send the instruction context to the cold context cache (operation 212). Note that the DMA processor does not directly write to the cold context cache. Therefore, to send the instruction context to the cold context cache, the processor may first write the instruction context to the hot context cache (e.g., at the location corresponding to the processed instruction) and then update the tracker entry state so that the tracker entry will generate a command (e.g., a HOT_2COLD command) for transferring the context from the hot context cache to the cold context cache. The instruction tracker entry can then compete for access to the SG DMA pipeline, where it generates the HOT_2COLD command. Tracker entries that urgently generate the HOT_2COLD command have the highest priority when accessing the SG DMA pipeline, where they generate and send the HOT_2COLD command to the cache system. Upon receiving the command, the cache system can transfer the instruction context from the hot context cache to the entry at the address corresponding to the message ID (e.g., a RAM address) in the path to the cold context cache.

[0041] In some respects, a cold context cache may include a four-way cache and may implement a Least Recently Used (LRU) cache replacement policy. When a context is transferred from the hot context cache to the cold context cache, the caching logic can determine whether an existing non-free entry exists at the target cache index (e.g., due to a cache conflict). If it does, the caching logic can determine whether a free entry exists at the target index. If a free entry exists, the caching logic can place the context from the hot context cache into the free entry in the cold context cache. If no free entry exists, the LRU entry can be replaced. After the context is transferred to the cold context cache, entries in the hot context cache can be invalidated.

[0042] In some aspects, message instructions are processed sequentially, and message completion is inferred when an instruction marked as the end of the message (indicating message processing completion) has been processed. In this scenario, the cold context buffer can be a single-way buffer, where the packet sequence number is implicit because the instructions for each message are processed sequentially. Alternatively, message instructions can be processed out of order, and message processing completion is indicated by an external command (e.g., a message deallocation command) input to the SG DMA engine with the transfer message ID. In this scenario, the cold context buffer can be a multi-way buffer (e.g., a four-way buffer), and the buffer context should be identified by both the message ID and the packet sequence number. Each of the four-way buffers may include RAM, and buffer reads involve reading all paths in parallel using the address obtained from the message ID. Each buffer entry contains a key consisting of the message ID and the packet sequence number.

[0043] When message processing is known to be complete (either via a message end marker or a message deallocation command), the caching logic can remove the message-associated entry from both caches (the hot context cache and the cold context cache). The associated ICB entries (including IOWEC entries) referenced by the removed context should also be released for reuse.

[0044] When instructions traverse the SG DMA pipeline (e.g., Figure 1 When the pipeline 106 shown is executed, it can be (e.g., by...) depending on the type of instruction and the state of the tracker entry corresponding to the instruction. Figure 1 The SG DMA processor 108 shown generates different context cache commands and sends these commands to a two-level context cache system (which includes a hot context cache and a cold context cache). In some aspects, there may be five different types of context cache commands, such as commands for reading context from the hot context cache (e.g., the HOT_READ command), commands for reading context from the cold context cache (e.g., the COLD_READ command), commands for evicting context from the hot context cache (e.g., the HOT_EVICT command), commands for transferring context from the hot context cache to the cold context cache (e.g., the HOT_2COLD command), and commands for eliminating (and evicting if necessary) any cached cold context referenced by a given message ID (e.g., the DEALLOC_M command).

[0045] The cache management logic can perform different cache operations based on the received commands. More specifically, the first set of commands can cause the cache management logic to read the hot context cache and then perform various cache-related operations, while the second set of commands can cause the cache management logic to read the cold context cache and then perform various cache-related operations. More specifically, if the context read from the cache is related to IVOEC processing (e.g., it references entries in the ICB buffer), operations related to the ICB entries (e.g., eviction and reference count updates) may be required. Note that the HOT_2COLD command (i.e., the command used to transfer context from the hot context cache to the cold context cache) can initiate a read from both the hot and cold context caches.

[0046] In some respects, the HOT_READ, HOT_EVICT, and HOT_2COLD commands can initiate read operations in the hot context cache. Figure 3A A flowchart illustrating an example process for processing various context cache commands for reading the hot context cache according to one aspect of this application is presented. During operation, the cache logic may receive a HOT_READ, HOT_EVICT, or HOT_2COLD command (operation 302) and read the hot context cache based on the received command (operation 304). The command may be generated by... Figure 1 The arbitrator 104 shown is selected from the instruction tracker 102 and can specify an address in the hot context cache (e.g., a cache index).

[0047] If the command is a HOT_READ command, the hot context cache can output the corresponding context (operation 306), and the caching system process ends. If the context read from the hot context cache references an ICB entry, the first ICB entry number in the list can be used to read the ICB buffer. If the command is a HOT_EVICT command, the caching logic can determine whether the context read from the hot context cache is an IVOEC processing context (operation 308). In one example, the caching logic can determine whether the context includes a list of entries in the ICB buffer that stores IVOEC entries previously read from host memory. If the context is not an IVOEC processing context (e.g., it could include a processing context of a derived data type containing nested loop counter values), the caching logic can invalidate the corresponding hot context cache entry after reading the hot context cache (operation 310). If the context read from the hot context cache references an ICB entry, additional actions are required to finally release the ICB entry for reuse. In one example, the list of ICB entries referenced by the context read from the hot context cache (referred to as hot ICB entries) can be transferred to the eviction queue (operation 312). Subsequently, the list of ICB entries in the eviction queue can be dequeued and the reference count associated with each listed entry in the ICB buffer can be updated (e.g., decremented) (operation 314). When the reference count of a given ICB entry reaches 0, the ICB entry is released for reuse.

[0048] If the received command is a HOT_2COLD command, in addition to specifying the location in the hot context buffer, the command can also specify a location within a path of the cold context buffer to transfer the context. In some examples, the RAM address in the cold context buffer path can be obtained from the message ID associated with the instruction that generated the HOT_2COLD command. Cache operations associated with the HOT_2COLD command are... Figure 3B Continued.

[0049] Figure 3B A flowchart illustrating an example process for transferring context from a hot context cache to a cold context cache, according to one aspect of this application, is presented. Figure 3B In the process of reading the hot context cache (i.e., Figure 3A After operation 304 shown, the caching logic can determine whether a matching entry (e.g., an entry with the same message ID and group sequence number) is stored at the target location in the cold context cache (operation 316).

[0050] If a matching entry is found at the target location in the cold context cache, the caching logic can further determine whether the entry contains an IVOEC processing context (operation 324). If the entry does not contain an IVOEC processing context, the caching logic can replace the matching entry with the context read from the hot context cache and invalidate the corresponding entry in the hot context cache (operation 326). If the entry contains an IVOEC processing context, the caching logic can transfer the referenced cold ICB entry to the eviction queue (operation 328). Subsequently, the list of ICB entries can be dequeued, and the reference count associated with each listed entry in the ICB buffer can be updated (operation 330). When the reference count of a given ICB entry reaches 0, the ICB entry is released for reuse.

[0051] If no matching entry is found, the caching logic can determine if any target location in the cache path has a free entry (operation 318). If a free entry exists, the caching logic can replace the free entry with the context read from the hot context cache and invalidate the corresponding entry in the hot context cache (operation 320).

[0052] If no free entry is found at the target location, the caching logic can select an LRU entry at the target location for replacement (operation 322). The caching logic can further determine whether the entry includes an IOWEC processing context (operation 324). If the entry does not include an IOWEC processing context, the caching logic can replace the LRU entry with the context read from the hot context cache and invalidate the corresponding entry in the hot context cache (operation 326). If the entry includes an IOWEC processing context, the caching logic can transfer the referenced cold ICB entry to the eviction queue (operation 328). Subsequently, the ICB entry can be dequeued, and the reference count in the ICB buffer can be updated (operation 330).

[0053] Figure 3C A flowchart illustrating an example process for processing various context cache commands for reading a cold context cache according to one aspect of this application is presented. During operation, the cache logic may receive a COLD_READ, DEALLOC_M, or HOT_2COLD command (operation 332) and read the cold context cache based on the received command (operation 334). This command can be executed in instruction 102. Figure 1 The arbitrator 104 shown is generated when selected, and the path address in the cold context cache can be specified.

[0054] If the command is a COLD_READ command, the caching logic can first determine whether a matching entry exists at the target location in the cold context cache (operation 336). For example, the caching logic can compare the message ID and packet sequence number (if applicable) specified by the received command with the message ID associated with the cold cache entry. If a matching entry is found, the cold context cache can simply output the corresponding context and invalidate the entry in the cold context cache (operation 338). If no matching entry is found, the caching logic can return a signal indicating that no context was found (operation 340).

[0055] If the command is a DEALLOC_M command (i.e., a command used to notify the SG DMA engine that message processing is complete), the cache logic can transfer any entry with a matching message ID at the target location to the eviction queue (operation 342). Subsequently, such an entry can be dequeued (i.e., evicted), and the corresponding entry in the cold context cache can be invalidated (operation 344).

[0056] If the command is a HOT_2COLD command, then after reading the cold context cache, the process can be based on... Figure 3B The operation shown continues.

[0057] although Figures 2 to 3C The example processes in the flowchart illustrate a specific order in which certain operations are performed, but these processes are not limited to this order. Operations shown consecutively in the flowchart may be performed in different orders, and may be performed simultaneously, partially simultaneously, or in combination.

[0058] Figure 4 An example network interface controller (NIC) according to one aspect of this application is illustrated. NIC 400 may be part of a computing node to provide network connectivity to the computing node. A computing node may include one or more NICs. In some examples, a computing node may include one, two, four, or eight NICs. NIC 400 may include more than... Figure 4 The entities shown are fewer or more entities. Figure 4 In the example shown, the NIC 400 may include at least a host interface (HI) 402 and a high-speed network interface (HNI) 404.

[0059] The HNI 402 may include a Peripheral Component Interconnect (PCI) or Peripheral Component Interconnect Fast (PCIe) interface and may be coupled to a host via a host connection having multiple channels (e.g., PCIe 4th generation channels capable of operating at signaling rates up to 25 Gbps per channel). The HNI 404 can facilitate high-speed network connectivity for communication with links in a switch architecture.

[0060] NIC 400 may include one or more processing resources (e.g., processing resource 406), one or more storage devices (e.g., storage device 408), and instruction processing system 410. In this example, storage device 408 may include volatile and non-volatile storage devices, and may further include a hot context cache 412 and a cold context cache 414. In some aspects, the hot context cache 412 and the cold context cache 414 may be implemented using RAM.

[0061] In the examples described herein, processing resources may include, for example, one or more processors included in a single computing device or distributed across multiple computing devices. As used herein, a “processor” may be at least one of a central processing unit (CPU), a semiconductor-based microprocessor, a graphics processing unit (GPU), a field-programmable gate array (FPGA) configured to fetch and execute instructions, other electronic circuitry suitable for fetching and executing instructions stored on a computer-readable storage medium, or combinations thereof. In the examples described herein, processing resources may fetch, decode, and execute instructions stored on a storage medium (e.g., storage device 408) to perform the functions described with respect to the instructions stored on the computer-readable medium. In other examples, the functions described with respect to any instructions described herein may be implemented in the form of electronic circuitry, executable instructions encoded on a computer-readable medium, or a combination thereof. The computer-readable storage medium may be located in or away from the computing device that executes the instructions but is accessible (e.g., via a computer network) for execution.

[0062] Instruction processing system 410 may include any number of software, hardware, and firmware functions that work together to achieve the goal of managing a two-level cache system including a hot context cache and a cold context cache. In some aspects, instruction processing system 410 may include computer functions that, when executed by processing resource 406, enable processing resource 406 to perform the methods and / or processes described in this disclosure. Specifically, instruction processing system 410 may include methods for receiving SG DMA instructions associated with messages (as described above regarding...). Figure 2 Function 416 (as described in operation 202). In some aspects, the message may be an MPI message, which may be associated with multiple packets. Each packet corresponds to an SG DMA instruction. In a further aspect, the SG DMA instruction may include a message ID and a packet sequence number.

[0063] Instruction processing system 410 may include an instruction context for aggregating SG DMA instructions (as described above regarding...). Figure 2The function 418 (described in operation 204) is shown in the diagram. More specifically, the instruction context may include memory address information associated with the instruction. In one example, the instruction context may include a list of entries in an ICB buffer, each ICB entry containing a set of IOWEC entries previously read from host memory, wherein the IOWECs in host memory include a list of memory addresses and sizes. In another example, the instruction context may include pointers, and the ICB buffer entries include data type context fast-forward information, which includes up to three nested loop count values ​​and a byte offset within the starting data element.

[0064] Instruction processing system 410 may include instructions for processing SG DMA instructions (as described above) based on the aggregated instruction context. Figure 2 Function 420 (described in operation 206) can be performed. Multiple DMA read / write operations generated by SG DMA instructions can be executed. For example, in the outbound direction, a read request can be issued to the host memory using address information identified by the instruction context, and in the inbound direction, the payload of the received packet can be written to various locations in the host memory based on the instruction context.

[0065] Instruction processing system 410 may include an address corresponding to the pending SG DMA instruction (as described above) for sending the instruction context to the hot context cache in response to the existence of a pending instruction. Figure 2 Function 422 (described in operation 210) is shown. More specifically, a pending instruction refers to an instruction in instruction tracker 102 that has the same message ID and subsequent group sequence number as a processed instruction. The cache index may correspond to the tracker entry number of the pending instruction.

[0066] Instruction processing system 410 may include a method for sending instruction context to a cold context buffer in response to the absence of pending instructions (as described above). Figure 2 Function 424 (as shown in operation 214). When there are no pending instructions, the context should be cached in the cold context cache. Because the SG DMA processor does not write directly to the cold context cache, the processor can first write the context to the location in the hot context cache corresponding to the processed instruction (e.g., according to its tracker entry number), and then generate a HOT_2COLD command to transfer the context to the location in the cold context cache corresponding to the message ID and optionally the packet sequence number specified by the instruction.

[0067] Instruction processing system 410 may include methods for processing pending SG DMA instructions (as described above) by retrieving the instruction context from a hot context cache. Figure 2Function 426 (as shown in operation 212). The SG DMA processor can access the cache location corresponding to the pending instructions (e.g., based on its tracker entry number) to obtain the instruction context.

[0068] NIC 400 can include more Figure 4 The entities shown are fewer or more entities. For example, NIC 400 may include functionality for invalidating / eviction of an entry in a hot context cache or a cold context cache. NIC 400 may further include functionality for removing ICB entries referenced by the evicted cache entry.

[0069] Figure 5 The illustration depicts a computer-readable medium facilitating the operation of a two-level caching system according to one aspect of this application. CRM 500 can be a non-transitory computer-readable medium or device storing computer functions that, when executed by a computer or processing resource, cause the computer or processing resource to perform a method. As used herein, a "computer-readable storage medium" can be any electronic, magnetic, optical, or other physical storage device for containing or storing information such as executable code, data, etc. For example, any computer-readable storage medium described herein can be any or a combination of RAM, EEPROM, volatile memory, non-volatile memory, flash memory, storage drives (e.g., HDD, SSD), any type of storage disk (e.g., compact disk, DVD, etc.). Furthermore, any computer-readable storage medium described herein can be non-transitory.

[0070] The CRM 500 can store SG DMA instructions for receiving messages (as mentioned above). Figure 2 Code 510 (as described in operation 202); instruction context for aggregating SG DMA instructions (as described above regarding...) Figure 2 Code 520 (as described in operation 204) is used to process SG DMA instructions based on the aggregated instruction context (as described above regarding...). Figure 2 Code 530 (as described in operation 206) is used to send the instruction context to the address corresponding to the pending SG DMA instruction in the hot context buffer in response to the existence of a pending instruction (as mentioned above regarding...). Figure 2 Code 540 (as described in operation 210) is used to send the instruction context to a cold context buffer in response to the absence of pending instructions (as described above regarding...). Figure 2 Code 550 (as described in operation 214); and code 550 for handling pending SG DMA instructions by retrieving the instruction context from the hot context cache (as described above regarding...). Figure 2 The code 560 (described in operation 212 shown) is as follows.

[0071] CRM 500 can store more Figure 5 The code shown is further code. For example, the CRM 500 can store code for invalidating / eviction of an entry in a hot or cold context cache, as well as code for removing an ICB entry referenced by the evicted cache entry.

[0072] Typically, the aspects of this disclosure address the technical problem of efficiently caching instruction context when processing SG DMA instructions. A NIC performing SG DMA operations may include a two-level context cache system, comprising a hot context cache for storing urgently needed instruction contexts (i.e., hot contexts) and a cold context cache for storing instruction contexts that may not be needed for a period of time (i.e., cold contexts). More specifically, after processing instructions associated with a message (e.g., an MPI message), the SG DMA processor can determine if there are any pending instructions for the same message. If so, the processor can write the instruction context to the address in the hot context cache corresponding to the pending instruction. If not, the process can transfer the instruction context to the address in the cold context cache corresponding to the message ID. After processing all instructions associated with the message, cache entries associated with the message and ICB entries referenced by the context can be removed. If the processor pauses instruction processing, the processor can write the instruction context to the location in the hot context cache corresponding to the instruction. When the processor resumes instruction processing, the processor can retrieve the instruction context from the hot context cache.

[0073] One aspect of this disclosure provides a system and method for processing distributed aggregated direct memory access (SG DMA) instructions. During operation, the system may receive an SG DMA instruction associated with a message and aggregate the instruction context of the SG DMA instruction. The SG DMA processor may process the SG DMA instruction based on the aggregated instruction context and determine whether there are any pending SG DMA instructions associated with the message. In response to the existence of a pending DMA instruction, the system stores the instruction context in a hot context cache at the address corresponding to the pending DMA instruction. In response to the absence of a pending DMA instruction, the system stores the instruction context in a cold context cache.

[0074] In a variation of this, storing the instruction context in a cold context cache may include: writing the instruction context to the address in the hot context cache corresponding to the processed SG DMA instruction; and transferring the instruction context from the hot context cache to the cold context cache.

[0075] In this variation, a hot context cache may include a single-way cache, and a cold context cache may include a four-way cache.

[0076] In a further variation, transferring the instruction context from the hot context cache to the cold context cache may include replacing the existing entry at the target location in the cold context cache.

[0077] In this variant, the system can process pending SG DMA instructions based on the instruction context stored in the hot context cache.

[0078] In this variant, the system can receive additional instructions associated with the message, retrieve the instruction context stored in the cold context cache, and process the additional instructions based on the instruction context retrieved from the cold context cache.

[0079] In this variant, the corresponding SG DMA instruction includes a message identifier and a packet sequence number corresponding to the message.

[0080] In a variation of this, the instruction context may include a list of entries in a buffer, each buffer entry storing a set of input / output vector (IOVEC) entries and / or data type context fast-forward information.

[0081] In a variation of this, the SG DMA processor can pause processing of SG DMA instructions, write the instruction context to the address in the hot context cache corresponding to the paused SG DMA instruction, and resume processing of SG DMA instructions by retrieving the instruction context from the hot context cache.

[0082] In a variation of this, in response to determining that the processed SG DMA instruction is the last instruction associated with the message or receiving an external command indicating that the processing of the message has been completed, the system can remove the entry associated with the message from both the hot context cache and the cold context cache.

[0083] One aspect of this disclosure provides an SG DMA engine for processing distributed aggregated direct memory access (SG DMA) instructions. The SG DMA engine may include: an SG DMA pipeline for aggregating instruction contexts of SG DMA instructions associated with a message; an SG DMA processor for processing SG DMA instructions based on the aggregated instruction context; a hot context cache; and a cold context cache. After processing the SG DMA instructions, the SG DMA processor stores the instruction context in the hot context cache at the address corresponding to the pending DMA instruction in response to determining the existence of a pending DMA instruction associated with the message, and stores the instruction context in the cold context cache in response to determining that no pending DMA instruction exists.

[0084] One aspect of this disclosure provides a network interface controller comprising: a host interface for receiving messages from and transmitting messages to a host; a network interface; and a distributed aggregated direct memory access (SG DMA) engine for processing SG DMA instructions associated with messages. The SG DMA engine may include: an SG DMA pipeline for aggregating instruction contexts of SG DMA instructions associated with messages; an SG DMA processor for processing SG DMA instructions based on the aggregated instruction contexts; a hot context cache; and a cold context cache. After processing the SG DMA instructions, the SG DMA processor stores the instruction context in the hot context cache at the address corresponding to the pending DMA instruction in response to determining the existence of a pending DMA instruction associated with the message, and stores the instruction context in the cold context cache in response to determining the absence of a pending DMA instruction.

[0085] In this disclosure, the functionality includes multiple logical units capable of performing predetermined logical functions described throughout this disclosure. Figure 2 The functions shown in Figure 3 can be implemented using any form of hardware, software, or a combination thereof. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software routines, or other mechanisms can be implemented to form the circuit. In implementations, the various functions described herein can be implemented as discrete circuits, or the described functions and features can be shared partially or wholly among one or more circuits. Although the various features or elements of a function can be described or declared as separate functions, these features and functions can be shared among one or more common functions, and such description does not require or imply the need for separate circuits to implement such features or functions.

[0086] The methods and processes described in the Detailed Description section can be embodied in code and / or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on the computer-readable storage medium, the computer system executes the methods and processes embodied in data structures and code and stored in the computer-readable storage medium.

[0087] The methods and processes described above may be included in hardware modules or devices. Hardware modules or devices may include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute specific software modules or code at specific times, and other programmable logic devices now known or developed later. When a hardware module or device is activated, it executes the methods and processes included therein.

[0088] The foregoing description is presented to enable any person skilled in the art to make and use the aspects and examples, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects and applications without departing from the spirit and scope of this disclosure. Therefore, the aspects described herein are not limited to those shown, but are intended to be consistent with the maximum scope of the principles and features disclosed herein.

[0089] Furthermore, the foregoing descriptions of the various aspects have been presented solely for illustrative and descriptive purposes. These descriptions are not intended to be exhaustive or to limit the aspects described herein to the disclosed forms. Accordingly, many modifications and variations will be apparent to those skilled in the art. Additionally, the foregoing disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.

Claims

1. A method comprising: Receive distributed aggregated direct memory access (SGDMA) instructions associated with the message; Aggregate the instruction context of the SG DMA instructions; The SG DMA instructions are processed by the SG DMA processor based on the aggregated instruction context; Determine if there are any pending SG DMA instructions associated with the message; In response to the existence of the pending SG DMA instruction, the instruction context is stored in the hot context cache at the address corresponding to the pending SG DMA instruction; as well as In response to the absence of the pending SG DMA instruction, the instruction context is stored in a cold context cache.

2. The method as described in claim 1, wherein, Storing the instruction context in the cold context cache includes: Write the instruction context to the address corresponding to the processed SG DMA instruction in the hot context cache; and The instruction context is transferred from the hot context cache to the cold context cache.

3. The method as described in claim 1, wherein, The hot context cache includes a single-way cache, and the cold context cache includes a four-way cache.

4. The method of claim 3, wherein, Transferring the instruction context from the hot context cache to the cold context cache includes replacing the existing entry at the target location in the cold context cache.

5. The method of claim 1, further comprising processing the pending SG DMA instruction based on the instruction context stored in the hot context cache.

6. The method of claim 1, further comprising: Receive additional instructions associated with the message; Retrieve the instruction context stored in the cold context cache; as well as The additional instructions are processed based on the instruction context obtained from the cold context cache.

7. The method of claim 1, wherein, The corresponding SG DMA instruction includes a message identifier and a packet sequence number corresponding to the message.

8. The method of claim 1, wherein, The instruction context includes a list of entries in a buffer, each buffer entry storing a set of input / output vector (IOVEC) entries and / or data type context fast-forward information.

9. The method of claim 1, further comprising: The SG DMA processor suspends the processing of the SG DMA instructions; Write the instruction context to the address in the hot context cache corresponding to the suspended SG DMA instruction; as well as The processing of the SG DMA instruction is resumed by retrieving the instruction context from the hot context cache.

10. The method of claim 1, further comprising: In response to determining that the processed SG DMA instruction is the last instruction associated with the message or receiving an external command indicating that the processing of the message has been completed, the entry associated with the message is removed from the hot context cache and the cold context cache.

11. An SG DMA engine for processing distributed aggregated direct memory access (SG DMA) instructions, the SG DMA engine comprising: The SG DMA pipeline is used to aggregate the instruction context of SG DMA instructions associated with a message; An SG DMA processor for processing the SG DMA instructions based on the aggregated instruction context; Hot context caching; as well as Cold context caching; After processing the SG DMA instructions, the SG DMA processor is used to: In response to determining that there is a pending SG DMA instruction associated with the message, the instruction context is stored in the hot context cache at the address corresponding to the pending SG DMA instruction; as well as In response to determining that there is no pending SG DMA instruction, the instruction context is stored in the cold context cache.

12. The SG DMA engine as claimed in claim 11, wherein, The SG DMA processor stores the instruction context in the cold context cache through the following operations: Write the instruction context into the hot context cache at the address corresponding to the processed SG DMA instruction; as well as The instruction context is transferred from the hot context cache to the cold context cache.

13. The SG DMA engine as described in claim 12, wherein, When transferring the instruction context from the hot context cache to the cold context cache, the SG DMA processor replaces existing entries in the cold context cache.

14. The SG DMA engine as claimed in claim 11, wherein, The SG DMA processor is further configured to process the pending SG DMA instructions based on the instruction context stored in the hot context cache.

15. The SG DMA engine as claimed in claim 11, wherein, The corresponding SG DMA instruction includes a message identifier and a packet sequence number corresponding to the message.

16. The SG DMA engine as claimed in claim 11, wherein, The instruction context includes a list of entries in a buffer, each buffer entry storing a set of input / output vector (IOVEC) entries and / or data type context fast-forward information.

17. The SG DMA engine as claimed in claim 11, wherein, The SG DMA processor is used for: Suspend processing of the SG DMA command; Write the instruction context to the address in the hot context cache corresponding to the suspended SG DMA instruction; as well as The processing of the SG DMA instruction is resumed by retrieving the instruction context from the hot context cache.

18. The SG DMA engine as claimed in claim 11, wherein, The SG DMS processor is further used for: In response to determining that the processed SG DMA instruction is the last instruction associated with the message or receiving an external command indicating that the processing of the message has been completed, the entry associated with the message is removed from the hot context cache and the cold context cache.

19. A network interface controller, comprising: A host interface is used to receive messages from a host and transmit messages to the host. Network interface; as well as A distributed clustered direct memory access (SG DMA) engine for processing S-GDMA instructions associated with the message, the SG DMA engine comprising: The SG DMA pipeline is used to aggregate the instruction context of SG DMA instructions associated with a message; An SG DMA processor for processing the SG DMA instructions based on the aggregated instruction context; Hot context caching; and Cold context caching; After processing the SG DMA instructions, the SG DMA processor is used to: In response to determining the existence of a pending SG DMA instruction associated with the message, the instruction context is stored in the hot context cache at the address corresponding to the pending SG DMA instruction; and In response to determining that there is no pending SG DMA instruction, the instruction context is stored in the cold context cache.

20. The network interface controller as claimed in claim 19, wherein, The SG DMA processor stores the instruction context in the cold context cache through the following operations: Write the instruction context into the hot context cache at the address corresponding to the processed SG DMA instruction; as well as The instruction context is transferred from the hot context cache to the cold context cache.

Images