Hardware Coherent Computation Extended Memory

Abstract

Embodiments herein describe transferring ownership of data (e.g., a cache line or a data block containing multiple cache lines) from a host to hardware in an I / O device. In one embodiment, the host and I / O device (e.g., an accelerator) are part of a cache coherent system, and ownership of data can be transferred from a home agent (HA) in the host to a local HA, e.g., a computational slave agent (CSA), in the I / O device. This allows functions on the I / O device (e.g., an accelerator function) to request data from the local HA without these requests having to be sent to the host HA. Furthermore, the accelerator function can indicate whether the local HA should track data on a cache line basis or by data block (e.g., multiple cache lines). This provides flexibility that can reduce the overhead from tracking data depending on the function's desired use of the data.

Description

[Technical Field]

[0001] Examples of the present disclosure generally relate to transferring ownership of data from a host to an I / O device (eg, an accelerator) in a cache coherent system. [Background technology]

[0002] Server CPU-accelerator systems such as Compute Express Link (CXL), Cache Coherent Interconnect for Accelerators (CCIX), QuickPath Interconnect (QPI) / Ultra Path Interconnect (UPI), AMD Infinity Fabric, NVLink, and those enabled by OpenCAPI are all inherently hardware cache coherent systems, meaning the hardware maintains a universal, coherent view of accessed, modified, and cached data regardless of whether the processor or accelerator is acting as a producer or consumer of the data and metadata (information about the data).

[0003] Current shared memory host-accelerator execution frameworks rely on fine-grained hardware coherency for producer-consumer interactions in their systems. Over time, at least one of the CPUs on the host or attached accelerator acts as a producer or consumer of data or metadata as part of an application or executing function. That data movement between producer-consumer pairs is tracked at a fine-grained cache-line level by a hardware coherency mechanism on either the host or accelerator, regardless of whether the producer-consumer action between the host CPU and accelerator is at a coarse-grained block level. This fine-grained tracking also occurs regardless of whether the home node and memory are adjacent to the host CPU producer / consumer or the accelerator producer / consumer.

[0004] Hardware coherency-based producer-consumer interactions rely on fine-grained tracking of data and metadata movement between producers and consumers. Producer and consumer actions on data and metadata typically occur at cache-line (e.g., 64-byte) granularity, e.g., MB, GB, or even TB granularity, regardless of whether the size of the data is several orders of magnitude larger than a cache line. A drawback of implementing fine-grained hardware coherency is that fine-grained tracking, which is independent of the size of producer and consumer actions on data and metadata, results in inefficient use of resources, both in terms of resources (such as caches and snoop filters) that track this data movement at a fine granularity and the coherency messaging overhead to track data and metadata movement and for the system to maintain a universal, coherent view of accessed, modified, and cached data. Hardware coherency calculation and messaging overhead at the home node also impacts the ability to maintain coherency in one region of memory where fine-grained tracking is required, because concurrent coherency actions are occurring on another region of memory where producer and consumer actions require coarse-grained tracking.Therefore, in current cache coherent systems, (i) the host unnecessarily tracks accelerator coherency actions to accelerator-attached memory managed by the host at a fine granularity; (ii) there is unnecessary data movement between the host and the accelerator that consumes bandwidth and queuing resources, e.g., accelerator accesses to local accelerator-attached memory must loop through the host because the memory is managed by the host; (iii) if the accelerator has a cache and the accelerator-attached memory also has a cache, there is inefficient use of caching resources for accelerator coherency actions to accelerator-attached memory; and (iv) accelerator coherency actions to accelerator-attached memory managed by the host impact the performance of other host CPU-to-host memory coherency actions. Summary of the Invention

[0005] One embodiment describes a hardware accelerator. The hardware accelerator includes a computational slave agent (CSA) communicatively coupled to accelerator function circuitry within the hardware accelerator. The CSA is configured to communicatively couple to a home agent (HA) of a host. The CSA and the HA are part of the same coherent domain. The CSA is configured to request ownership of a data set from the HA so that the CSA acts as a local HA for the data set. The accelerator function circuitry indicates whether the CSA tracks the data set at a cache line level or a data block level when acting as the local HA.

[0006] Another embodiment describes a computing system including a host including a request agent (RA) and a home agent (HA), and an input / output (I / O) device communicatively coupled to the host, the I / O device including a computational slave agent (CSA) communicatively coupled to the I / O device and an accelerator function within the HA, the CSA and the HA being part of the same coherent domain. Further, the CSA is configured to request ownership of a data set from the HA such that the CSA acts as a local HA for the data set, and the accelerator function indicates whether the CSA tracks the data set at a cache line level or a data block level when acting as the local HA.

[0007] Another embodiment described herein is a method that includes communicatively coupling a host including an RA and an HA to an I / O device, the I / O device including an accelerator function within the I / O device and a CSA communicatively coupled to the HA, where the CSA and the HA are part of the same coherent domain, and using the CSA to request ownership of a data set from the HA, such that the CSA acts as a local HA for the data set. Further, the accelerator function indicates whether the CSA, when acting as the local HA, will track the data set at a cache line level or a data block level.

[0008] Another embodiment described herein is an I / O device that includes an accelerator function, a CSA communicatively coupled to the accelerator function, and an HA in a host, wherein the CSA and the HA are part of the same coherent domain. The CSA is configured to request ownership of a data set from the HA such that the CSA acts as a local HA for the data set, and the accelerator function indicates whether the CSA tracks the data set at a cache line level or a data block level when acting as the local HA. [Brief explanation of the drawings]

[0009] So that the above-enumerated features briefly summarized above may be understood in detail, a more particular description can be had by reference to exemplary implementations, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical exemplary implementations and therefore should not be considered limiting of the scope thereof. [Figure 1] 1 is a block diagram of a host coupled to an accelerator device, according to an example. [Figure 2] 1 is a flowchart illustrating transferring ownership of data to hardware in an I / O device, according to an example. [Figure 3] 1 is a flowchart for transferring ownership of data from hardware in an I / O device to a host, according to an example. [Figure 4] 10 is a flow chart for updating a host home agent when changing ownership from a shared state to an exclusive state, according to an example. [Figure 5] 10 is a timing diagram for transferring ownership of a cache line to hardware in an I / O device, according to an example. [Figure 6] 10 is a timing diagram for transferring ownership of multiple cache lines to hardware in an I / O device, according to an example. [Figure 7] 1 is a timing diagram for transferring ownership of a block of data to hardware within an I / O device, according to an example. [Figure 8] 1 illustrates a field programmable gate array implementation of a programmable IC, according to an example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Various features are described below with reference to the drawings. It should be noted that the drawings may or may not be drawn to scale, and that elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should be noted that the drawings are intended only to facilitate the description of the features. They are not intended as an exhaustive description of the description or as a limitation on the scope of the claims. In addition, the illustrated example need not have all aspects or advantages shown. An aspect or advantage described in connection with a particular example is not necessarily limited to that example and may be implemented in any other example, even if not so shown or explicitly described.

[0011] Embodiments herein describe the transfer of ownership of data (e.g., a cache line or a block of data comprising multiple cache lines) from a host to hardware in an I / O device. In one embodiment, the host and I / O device (e.g., an accelerator device) are part of a cache-coherent system such as those enabled by CXL, CCIX, QPI / UPI, AMD Infinity Fabric, NVLink, and OpenCAPI. To mitigate or overcome the above-mentioned drawbacks, the following techniques can transfer ownership of data from a home agent (HA) in the host to a local HA, e.g., a compute slave agent (CSA), in the I / O device. This allows functions on the I / O device (e.g., accelerator functions) to request data from the local HA without these requests having to be sent to the host HA. Furthermore, the accelerator function can indicate whether the local HA should track data at the cache line level or the data block level. This provides flexibility that can reduce overhead from tracking data depending on the accelerator function's desired use of the data. Furthermore, ownership of the data can be shared by the host HA and the local HA, which are in the same cache-coherent domain. For example, a request agent (RA) on a host can maintain a read-only copy of the data for reference, while a local HA allows an accelerator function on an I / O device to access the data without sending a request to the host HA.

[0012] 1 is a block diagram of a hardware cache coherent computing system 100 including a host 105 coupled to an I / O device 130, according to an example. In computing system 100, host 105 (e.g., a server or other computing device or system) is communicatively coupled to I / O device 130 (e.g., an accelerator device, a field programmable gate array (FPGA), a graphics processing unit (GPU), an integrated circuit, a printed circuit board (PCB), or a system on a chip (SOC)) via a PCIe connection 125. In one embodiment, I / O device 130 is integrated into host 105 (e.g., a PCB card connected to a PCIe slot), or device 130 may be external to host 105.

[0013] The host 105 includes at least one RA 110 coupled to an HA 115 (referred to as host HA). Although not shown, the host 105 may include any number of processors, which may include any number of processing cores. The host 105 may also include memory having volatile memory elements, non-volatile memory elements, and combinations thereof. For example, a processor (or a software application running on the processor) may be an RA 110 that performs read and write transactions to different addresses in memory. The HA 115 is responsible for a range of memory addresses in the system. In other words, the HA 115 has ownership of memory addresses within the range so that cache coherency is maintained. If the RA 110 wants to modify data owned by the HA 115, the HA 115 must approve the request. Furthermore, if the RA 110 wants to read data, the HA 115 ensures that the RA 110 is provided with the latest copy.

[0014] The I / O device 130 includes multiple CSAs 135 coupled to accelerator functions 150. That is, the accelerator functions 150 can use the CSAs 135 to read and modify data tracked or owned by the host HA 115. For example, the accelerator functions 150 (e.g., machine learning functions, cryptographic functions, compression functions, network functions, etc. implemented on hardware processing elements or programmable logic) can send requests to read or write data to the CSAs 135, which forwards these requests to the HA 115. In many cases, the requested data is stored in memory within the I / O device 130. Therefore, it is advantageous for the data requested by the accelerator functions 150 to be owned (or tracked) by a local HA within the I / O device 130, such as one of the CSAs 135, rather than by the host HA 115.

[0015] As shown, the CSA 135 includes an RA 140 coupled to one or more slave agents (SAs) 145. The CSA 135 can have at least three different states or modes of operation. In a first state, the CSA 135 does not track or own the data. For example, the I / O device 130 may be used by the host 105 as extended memory where data is stored in memory on the I / O device 130, but the I / O device 130 does not process the data (e.g., an accelerator function may not be operating on the data). This may be referred to as a disabled state, as the CSA 135 does not function as a local HA for the data. In another state, the CSA 135 functions as a local HA, but is in a shared state. In this state, the CSA 135 and the host HA 115 can share ownership of the data. That is, both the CSA 135 and the host HA 115 can track the data. An example of this situation is when both the RA 110 in the host 105 and the RA 140 in the I / O device 130 are requesting data.

[0016] In a third state, the CSA 135 owns the data as the local HA, but the host HA 115 does not (referred to as an exclusive state). In one embodiment, the host HA 115 has no record that the data is owned by the CSA 135. In other words, the host HA 115 does not need to track the transfer of ownership data to the CSA 135. This further reduces overhead in the HA 115. As shown, the HA 115 includes a tracking data list 120 that indicates the memory addresses of data that it owns or tracks. When transferring ownership of data to the CSA 135, the memory address of the transferred data may be deleted from the tracking data list 120. Thus, when the host RA 110 requests data, the HA 115 does not know where the data is, but sends a request (e.g., a snoop) to the CSA 135 to notify the CSA 135 that it desires ownership. The CSA 135, now acting as the local HA for the data, then returns ownership to the host HA 115.

[0017] 2 is a flowchart of a method 200 for transferring ownership of data to hardware in an I / O device (e.g., a CSA), according to an example. Method 200 begins assuming that a host HA owns or tracks data (e.g., a data set including one or more cache lines), which may include one or more cache lines. At block 205, the CSA, or more specifically, an RA in the CSA, receives a request to process the data from an accelerator function in the I / O device. The accelerator function also indicates whether the CSA should become the local HA for the data (e.g., whether the CSA should transition from an invalid state to either a shared state or an exclusive state).

[0018] Additionally, the accelerator function can indicate whether the CSA should track data at the cache line level or the block level. The accelerator function is the orchestrator that reads and / or modifies the data and therefore knows whether a particular application would benefit more from a CSA that tracks each cache line individually or multiple cache lines as a block of data. That is, one application may benefit more from a CSA that tracks data at the cache line level, while another application may benefit more from a CSA that tracks data at the block level. In one embodiment, the accelerator function can execute code or have programmable logic that represents patterns of behavior that would benefit more from tracking at the cache line level or tracking at the block level.

[0019] In block 210, the CSA or RA in the I / O device sends a cache maintenance operation (CMO) to the host HA. In one embodiment, the CMO indicates that the host HA should transfer ownership of the data indicated in the CMO to the CSA. For example, the CMO may be a clean invalidation operation in which the host HA identifies and pushes the latest version of the requested data (which may be stored in one or more local caches in the host, such as a last level cache or buffer) to the CSA.

[0020] In block 215, the CSA determines whether the data should be shared with the RA in the host. For example, the host RA may want to keep a copy of the data. In that scenario, the accelerator function may be the producer and is performing calculations on the data. Whether the accelerator function modifies the data or the host has a cached copy, switching the CSA to a shared state allows the host RA to continue to reference its local cached copy (e.g., a read-only data structure for fast access). When the CSA is in a shared state, unlike when it is in an exclusive state, the host HA tracks the fact that the data is shared by the RAs in the host and the I / O device.

[0021] If the CSA owns the data in an exclusive state, method 200 proceeds to block 220, where the host HA invalidates cached copies of the data in the host. In one embodiment, the host HA issues a flush CMO to remove the cached copies. Additionally, although not shown in method 200, when flushing and invalidating cached copies of the data in the host, the host HA can also remove the data from its tracking data list because the local HA (i.e., the CSA) now owns the data in an exclusive state. Thus, the host HA no longer needs to track the data.

[0022] In block 225, the host HA sends a push CMO to push any buffered copies of the data to the SA in the CSA in the I / O device. However, pushing data to the SA in the I / O device may not guarantee that the data actually reaches the SA. For example, the data may be stuck in a last level cache or buffer. Therefore, the host HA can issue a new push CMO, CleanInvalidateDeep, which completes only after the CMO reaches the SA. Thus, this push CMO differs from a flush, which does not guarantee that the data reaches the SA in the I / O device.

[0023] In block 230, the CSA waits until the SA receives the push CMO. That is, the CSA monitors traffic from the RA to the SA to determine when the push CMO arrives at the SA. Once this occurs, in block 235, the CSA transitions to an exclusive state where the local HA owns and tracks the data at either the cache line level or the block level. As a result, the CSA can allow the local RA in the I / O device to read and modify the data without first obtaining permission from the host HA. Thus, when an accelerator function requests the data, the CSA can retrieve the data from memory and cache a local copy of the data for the accelerator function to read and modify in the I / O device.

[0024] Returning to block 215, if ownership of the data should instead be shared by the host HA and CSA (i.e., the local HA) rather than the CSA that owns the data in the exclusive state, method 200 proceeds to block 237, where the data in the host that should be shared with the local HA is changed to a shared state. In block 240, the host HA sends a push CMO to push any buffered copies of the data to the SA in the CSA. Notably, the host HA does not invalidate cached copies of the data in the host, in contrast to block 220. Furthermore, the host HA can push the data to the SA in the I / O device using the same technique as described in block 225.

[0025] In block 245, the CSA waits until it confirms that the SA has received the push CMO. Once confirmed, in block 250, the CSA transitions to a shared state, local HA. In this state, both the CSA and the host HSA track the data. For example, the host HA may track in its tracked data list that the data is in use by both an RA in the host and an RA in the I / O device. Because the host HA needs to track that the data is in use by an RA in the host, also tracking that the same data is in use by an RA in the I / O device adds very little overhead (e.g., one or more additional bits) to the host HA.

[0026] 3 is a flowchart of a method 300 for transferring ownership of data from hardware in an I / O device to a host, according to an example. Method 300 assumes that the CSA has transitioned to a local HA in either a shared or exclusive state, as described in method 200 of FIG. 2. When the CSA owns the data (partially or exclusively), the host HA receives a request from an RA in the host to modify the data. The host HA must again have exclusive ownership of the data before allowing the RA on the host to edit the data.

[0027] In block 305, the host HA receives a request from an RA in the host for data that is currently owned by the local HA in either a shared or exclusive state. If the data is owned by the local HA in a shared state, the host HA can know that the RA in the I / O device is also using the data. If the data is owned by the local HA in an exclusive state, the host HA cannot know which CSA is the local HA because, as described above, the host HA does not need to track the data after it is transferred to the local HA. In either case, the host HA can regain exclusive ownership of the data.

[0028] In block 310, the host HA sends a snoop to the local HA / CSA. The snoop notifies the local HA that the host HA wants to return ownership of the data. In response, the local HA can flush its local cache so that the latest version of the data is transferred to the host.

[0029] In block 315, the CSA / local HA transitions to a disabled state in which the CSA no longer functions as the local HA. Thus, requests from the RA in the I / O device for data are forwarded to the host HA rather than being serviced by the CSA. In this way, the host HA can regain ownership of the data from the local HA.

[0030] 4 is a flowchart of a method 400 for updating a host HA when changing ownership from a shared state to an exclusive state, according to an example. In block 405, the CSA transitions to a local HA in a shared state. That is, the CSA and the host HA share ownership of the same data. As a result, both the RA in the host and the RA in the I / O device can access the same data.

[0031] In block 410, the host HA tracks the fact that cached copies of the data are being used by the host RA and the RA in the I / O device. That is, the tracked data list in the host HA can indicate that both the host and the RA in the I / O device have cached copies of the data. Doing so allows, for example, an RA on the I / O device to modify the data while the host RA can retain a copy of the data for fast read-only data access. Because the host HA tracks both host and I / O device usage of the data, when the host RA sends a request to modify the data, the host HA knows that it must first invalidate and flush the copy of the data in the RA of the I / O device before allowing the host RA to modify the data.

[0032] However, in block 415, the host HA receives a CMO from the host RA that the host RA no longer needs access to its copy of the data. Therefore, the only RA that needs the data is the RA in the I / O device.

[0033] In block 420, the host HA updates the tracking data list to stop tracking the data in both the host RA and the I / O device RA, thereby reducing overhead in the host HA. Furthermore, the local HA can transition from the shared state to the exclusive state because the data is no longer being used by the RA in the host. If the RA in the host again requests use of the data, the system can use method 300 of FIG. 3, in which the host HA can again share ownership with the local HA (and the local HA transitions to the shared state).

[0034] 5 is a timing diagram for transferring ownership of a cache line to hardware in an I / O device, according to an example. In one embodiment, the timing diagram of FIG. 5 provides additional details corresponding to method 200 described in FIG. 2. The timing diagram begins with the accelerator function sending a data request to a CSA in the I / O device for data owned by the host HA. That is, the CSA is not currently acting as a local HA (i.e., the CSA is in an invalid state).

[0035] In this embodiment, the data request from the accelerator is for a specific cache line indicating that the CSA (after transitioning to the local HA in the exclusive state) should track the data at the cache line level. In response, the CSA issues a clean invalidate CMO to the host HA, which prompts the host HA to flush and invalidate cached copies of the cache line in the host. As part of this process, Figure 5 shows the host HA issuing a snoop to the host RA to invalidate and flush cached copies. The host HA may also issue a completion (comp) CMO to the CSA indicating that it has received the clean invalidate CMO.

[0036] After receiving comp from snoop CMO, the host HA issues a push CMO (i.e., Clean Invalidate Deep (CleanInvDeep)) to the CSA. As described above, when performing a flush, the flushed copy of the cache line may be stacked in the last level cache or buffer and may not reach the SA in the CSA. The push CMO ensures that the latest version of the cache line reaches the SA.

[0037] Upon confirming that CleanInvDeep has reached the SA, the CSA can transition to the local HA in an exclusive state, allowing local access of the cache line by the accelerator. The CSA can send a confirmation that it received the push CMO to the host HA. The host HA can stop tracking the cache line in its tracking data list. Additionally, using the timeline of Figure 5, the CSA can also transition to the local HSA in a shared state, but in that embodiment, cached copies in the host cannot be invalidated. That is, the CSA can send a CMO different from the clean invalidate to prevent the host HA from invalidating cached copies.

[0038] 5 also shows that the host HA again receives ownership of the cacheline, as described in method 300 of FIG. 3. After transferring ownership to the CSA, later, as shown by the timeline, the host RA requests access to the cacheline using a read-specific or write-specific request. For example, a processor function executing within the host may wish to read or modify the cacheline. Because the host HA is not tracking the cacheline, the host HA sends a snoop to the CSA instructing the CSA to send the latest version of the cacheline to the host HA. Additionally, the CSA transitions from functioning as a local HA to an invalidation state in which it is no longer functioning as a local HA.

[0039] FIG. 6 is a timing diagram for transferring ownership of multiple cache lines to hardware in an I / O device, according to an example. That is, unlike FIG. 5, in which the accelerator function requests ownership of a single cache line, the accelerator in FIG. 6 sends data requests for multiple cache lines, and the CSA must track the cache lines individually. In response, the CSA issues a CMO for each of the cache lines (i.e., lines 1 through N). As described above, the type of CMO changes depending on whether the CSA transitions to a shared or exclusive state. In the shared state, the host HA does not need to invalidate cached copies in the host.

[0040] Similar to above, the host HA issues one or more snoops to the host RA to retrieve the latest version of the data and sends a comp CMO to the CSA corresponding to each cache line. Upon receiving the comp CMO in response to the snoop, the host HA can send a push CMO (i.e., CleanInvDeep CMO) for cache lines 1 through N so that the cache lines are pushed to the SAs within the CSA. Upon receiving the push CMO for the last cache line (i.e., line N), the CSA can transition to running as a local HA in either a shared or exclusive state.

[0041] The host RA can then request access to the cache line using a read / write specific request. As described above, the host HA can regain ownership of the CSA, which transitions to the invalidated state (or the shared state if ownership is shared by both the local HA and the host HA).

[0042] 7 is a timing diagram for transferring ownership of a block of data to hardware in an I / O device, according to an example. In this embodiment, an accelerator sends a request for a block of data that includes multiple cache lines. That is, instead of the CSA tracking individual cache lines, FIG. 7 shows the accelerator transitioning to a local HA and instructing the CSA to track the corresponding data at the block level instead of the cache line level.

[0043] In response, the CSA sends a CMO for each cache line in the block (e.g., lines 1 to N). Again, the type of CMO changes if the accelerator instructs the host HA and CSA to share ownership of the data block, or if the data block is owned exclusively by the CSA. After sending the CMO for each cache line in the data block, the CSA, or more specifically, the RA in the CSA, sends a Clean Invalidate Block Deep (CleanInvBlockDeep) to the host HA. The CSA then begins monitoring the block cache state.

[0044] The host HA issues a snoop CMO to the host RA and a comp to the CSA, as described above. However, unlike Figure 6, in which the host HA issues a push CMO for each cacheline (after receiving a comp from the host RA), the host HA issues a push CMO CleanInvBlockDeep back to the CSA. In one embodiment, the CSA receives a CleanInvBlockDeep for each aggregated port (assuming the aggregated ports are available). Alternatively, the host HA issues a single CleanInvBlockDeep. After receiving the CleanInvBlockDeep, the CSA can transition to function as a local HA in either a shared or exclusive state.

[0045] Although not shown in FIG. 7, the host HA can regain partial or exclusive ownership of the data block using the techniques described above.

[0046] The above techniques provide some non-limiting advantages. For example, because the accelerator has transparently migrated coherency management from the host to the accelerator, the host HA no longer tracks accelerator coherency actions to the accelerator-attached memory managed by the host at a fine-grained level. Furthermore, host tracking resources, such as snoop filters, are also no longer unnecessarily consumed, freeing these resources to improve tracking resources for host-CPU-memory interactions using these same snoop filters. Furthermore, following the migration of coherency management from the host to the accelerator, the accelerator directly accesses the local accelerator-attached memory, eliminating unnecessary data movement between the host and the accelerator. Also, cache resources in the accelerator are used efficiently; the accelerator cache can act as both an accelerator cache and an accelerator-attached memory cache based on the use case and traffic patterns. Furthermore, accelerator coherency actions to accelerator-attached memory no longer affect the host, and therefore no longer affect the performance of coherency actions from other host CPUs to host memory.

[0047] 8 illustrates an FPGA 800 implementation of I / O device 130, and more specifically, an FPGA with a PL array containing a number of different programmable tiles, including transceivers 37, CLBs 33, BRAM 34, input / output blocks (“IOBs”) 36, configuration and clocking logic (“CONFIG / CLOCKS”) 42, DSP blocks 35, specialized input / output blocks (“IOs”) 41 (e.g., configuration ports and clock ports), and other programmable logic 39, such as digital clock managers, analog-to-digital converters, system monitoring logic, etc. The FPGA may also include a PCIe interface 40, an analog-to-digital converter (ADC) 38, etc.

[0048] In some FPGAs, each programmable tile may include at least one programmable interconnect element (“INT”) 43 with connections to input and output terminals 48 of programmable logic elements within the same tile, as shown by the example included at the top of FIG. 8 . Each programmable interconnect element 43 may also include connections to interconnect segments 49 of adjacent programmable interconnect elements within the same tile or other tiles. Each programmable interconnect element 43 may also include connections to interconnect segments 50 of general-purpose routing resources between logic blocks (not shown). The general-purpose routing resources may include routing channels between logic blocks (not shown) that comprise tracks of interconnect segments (e.g., interconnect segment 50) and switch blocks (not shown) for connecting the interconnect segments. The interconnect segments of the general-purpose routing resources (e.g., interconnect segment 50) may span one or more logic blocks. The programmable interconnect elements 43, together with the general-purpose routing resources, implement a programmable interconnect structure (“programmable interconnect”) for the illustrated FPGA.

[0049] In an exemplary implementation, CLB 33 may include a single programmable interconnect element (“INT”) 43 in addition to configurable logic elements (“CLEs”) 44 that can be programmed to implement user logic. BRAM 34 may include BRAM logic elements (“BRLs”) 45 in addition to one or more programmable interconnect elements. Typically, the number of interconnects included in a tile depends on the height of the tile. In the illustrated example, the BRAM tile has the same height as five CLBs, although other numbers (e.g., four) may also be used. DSP block 35 may include a DSP logic element (“DSPL”) 46 in addition to an appropriate number of programmable interconnect elements. IOB 36 may include, for example, one instance of programmable interconnect element 43 in addition to two instances of input / output logic element (“IOLs”) 47. As will be apparent to those skilled in the art, for example, the actual IO pads connected to IO logic element 47 are typically not limited to the area of ​​the input / output logic element 47.

[0050] In the illustrated example, a horizontal region near the center of the die (shown in FIG. 8) is used for configuration, clocks, and other control logic. Vertical columns 51 extending from this horizontal region or column are used to distribute clock and configuration signals across the width of the FPGA.

[0051] Some FPGAs utilizing the architecture shown in Figure 8 include additional logic blocks that interrupt the regular columnar structure that makes up the majority of the FPGA. The additional logic blocks may be programmable blocks and / or dedicated logic.

[0052] It should be noted that Figure 8 is intended to illustrate only an exemplary FPGA architecture. For example, the number of logic blocks in a row, the relative widths of the rows, the number and order of rows, the types of logic blocks included in the rows, the relative sizes of the logic blocks, and the interconnect / logic implementation included in the upper part of Figure 8 are purely exemplary. For example, in an actual FPGA, two or more adjacent rows of CLBs are typically included wherever a CLB appears to facilitate efficient implementation of user logic, although the number of adjacent CLB rows varies with the overall size of the FPGA.

[0053] In the foregoing, reference is made to embodiments presented in the present disclosure. However, the scope of the present disclosure is not limited to the specific described embodiments. Instead, any combination of the described features and elements, whether associated with different embodiments or not, is contemplated to implement and practice the contemplated embodiments. Furthermore, while the embodiments disclosed herein may achieve advantages over other possible solutions or prior art, whether or not a particular advantage is achieved by a given embodiment does not limit the scope of the present disclosure. Accordingly, the foregoing aspects, features, embodiments, and advantages are merely exemplary and are not considered elements or limitations of the appended claims unless expressly recited in the claims.

[0054] As will be appreciated by one skilled in the art, embodiments disclosed herein may be embodied as a system, method, or computer program product. Accordingly, aspects may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be generally referred to as a "circuit," "module," or "system." Furthermore, aspects may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied therein.

[0055] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this specification, a computer-readable storage medium is any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0056] A computer-readable signal medium may include a propagated data signal in which computer-readable program code is embodied, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium but may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0057] The program code embodied on the computer readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0058] Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or a connection to an external computer may be made (e.g., via the Internet using an Internet Service Provider).

[0059] Aspects of the present disclosure are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments presented in this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed by a processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0060] These computer program instructions may also be stored on a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other device to function in a particular way, such that the instructions stored on the computer-readable medium produce an article of manufacture including instructions that implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0061] Computer program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device to create a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide a process for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0062] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or a combination of dedicated hardware and computer instructions.

[0063] Some examples of the disclosed technology can be characterized in the following non-limiting examples.

[0064] Example 1. A hardware accelerator, comprising: an accelerator function circuit; a computational slave agent (CSA) communicatively coupled to the accelerator function circuitry and configured to communicate with a home agent (HA) in the host, wherein the CSA and the HA are part of the same coherent domain; the CSA is configured to request ownership of the data set from the HA such that the CSA acts as a local HA for the data set; The accelerator function circuit indicates whether the CSA tracks the data set at the cache line level or the data block level when acting as a local HA.

[0065] Example 2. The hardware accelerator of Example 1, wherein before the CSA requests ownership of the dataset from the HA, the accelerator function is configured to send a request to the CSA to perform at least one of reading or modifying the dataset, and the request from the accelerator function indicates whether the CSA is to track the dataset at a cache line level or a data block level.

[0066] Example 3. The hardware accelerator of example 2, wherein the HA is configured to, in response to a request from the CSA, perform a snoop to a request agent (RA) in the host to flush a cached copy of the data set from the host.

[0067] Example 4. The hardware accelerator of Example 3, wherein the HA is configured to send a push cache maintenance operation (CMO) to the CSA after receiving an indication that the snoop is complete, and the push CMO moves any cached copies of the data set stacked in the buffer or last level cache to a slave agent in the CSA.

[0068] Example 5. The hardware accelerator of example 4, wherein after determining that the slave agent has received the push CMO, the CSA transitions to act as a local HA for the data set.

[0069] Example 6. The hardware accelerator of Example 1, wherein the CSA can function as a local HA in one of (i) an exclusive state in which the HA in the host does not track the dataset, or (ii) a shared state in which both the CSA and the HA track the dataset and the RA in the host has a cached copy of the dataset.

[0070] Example 7. HA is receiving a request from an RA in the host to modify the data set after the CSA has begun its role as a local HA; and sending a request to the CSA to transition to a disabled state in which the CSA no longer functions as a local HA, and ownership of the data set is returned to the HA.

[0071] Example 8. A method comprising: Communicatively coupling a host including an RA and an HA to an I / O device, the I / O device including a CSA communicatively coupled to an accelerator function within the I / O device and the HA, the CSA and the HA being part of the same coherent domain; using the CSA to request ownership of the data set from the HA, such that the CSA acts as a local HA for the data set; The accelerator function indicates whether the CSA tracks the data set at the cache line level or the data block level when acting as a local HA.

[0072] Example 9. Before a CSA requests ownership of a dataset from an HA, 9. The method of claim 8, further comprising sending a request from the accelerator function to the CSA to perform at least one of reading or modifying the dataset, wherein the request from the accelerator function indicates whether the CSA tracks the dataset at a cache line level or a data block level.

[0073] Example 10. 10. The method of embodiment 9, further comprising using the HA to perform a snoop to the RA in response to a request from the CSA to flush a cached copy of the data set from the host.

[0074] Example 11 The method described in Example 10, further comprising sending a push CMO from the HA to the CSA after receiving an indication that the snoop is complete, the push CMO moving any cached copies of the data set stacked in the buffer or last level cache to a slave agent in the CSA.

[0075] Example 12: After determining that the slave agent has received a push CMO, 12. The method of example 11, further comprising transitioning the CSA to serve as a local HA for the dataset.

[0076] Example 13. The method described in Example 8, wherein the CSA can function as a local HA in one of (i) an exclusive state in which the HA in the host does not track the dataset, or (ii) a shared state in which both the CSA and the HA track the dataset and the RA in the host has a cached copy of the dataset.

[0077] Example 14. receiving, at the HA, a request from the RA to modify the data set after the CSA begins functioning as a local HA; The method of example 8, further comprising sending a request from the HA to the CSA to transition to a disabled state in which the CSA no longer functions as a local HA, and ownership of the dataset is returned to the HA.

[0078] Example 15. An I / O device configured to be communicatively coupled to a host, the device comprising: Accelerator function and a CSA communicatively coupled to an accelerator function and an HA within the host, the CSA and the HA being part of the same coherent domain; the CSA is configured to request ownership of the data set from the HA such that the CSA acts as a local HA for the data set; The accelerator capability indicates whether the CSA tracks data sets at the cache line level or data block level when acting as a local HA for an I / O device.

[0079] Example 16. The I / O device of Example 15, wherein before the CSA requests ownership of the dataset from the HA, the accelerator function is configured to send a request to the CSA to perform at least 15 reads or modifications of the dataset, and the request from the accelerator function indicates whether the CSA tracks the dataset at a cache line level or a data block level.

[0080] Example 17. The I / O device of Example 16, wherein the HA is configured to, in response to a request from the CSA, perform a snoop to the RA in the host to flush a cached copy of the data set from the host.

[0081] Example 18. The I / O device of Example 17, wherein the HA is configured to send a push CMO to the CSA after receiving an indication that the snoop is complete, and the push CMO moves any cached copies of the data set stacked in the buffer or last level cache to a slave agent in the CSA.

[0082] Example 19. The I / O device of example 18, wherein after determining that the slave agent has received the push CMO, the CSA transitions to act as a local HA for the data set.

[0083] Example 20. The I / O device of Example 15, wherein the CSA can function as a local HA in one of (i) an exclusive state in which the HA in the host does not track the dataset, or (ii) a shared state in which both the CSA and the HA track the dataset and the RA in the host has a cached copy of the dataset.

[0084] While the above is directed to particular examples, other and further examples may be devised without departing from the basic scope thereof, which scope is determined by the claims that follow.

Claims

1. A hardware accelerator, an accelerator function circuit; a Computation Slave Agent (CSA) communicatively coupled to the accelerator function circuitry within the hardware accelerator and configured to communicatively couple to a Home Agent (HA) of a host, wherein the CSA and the HA are part of the same coherent domain; the CSA is configured to request ownership of the data set from the HA such that the CSA acts as a local HA for the data set; The accelerator function circuit indicates whether the CSA tracks the data set at a cache line level or a data block level when functioning as the local HA.

2. 2. The hardware accelerator of claim 1, wherein before the CSA requests ownership of the data set from the HA, the accelerator function circuitry is configured to send a request to the CSA to at least one of read or modify the data set, the request from the accelerator function circuitry indicating whether the CSA tracks the data set at the cache line level or the data block level.

3. 3. The hardware accelerator of claim 2, wherein the HA is configured to respond to the request from the CSA by snooping to an RA to flush a cached copy of the dataset from the host, the RA being capable of storing a cached copy of the dataset.

4. 4. The hardware accelerator of claim 3, wherein the HA is configured to send a push cache maintenance operation (CMO) to the CSA after receiving an indication that the snoop is complete, the push CMO causing any cached copies of the data set stacked in a buffer or last level cache to be moved to a slave agent within the CSA.

5. 5. The hardware accelerator of claim 4, wherein after determining that the slave agent has received the push CMO, the CSA transitions to function as a local HA for the data set.

6. 2. The hardware accelerator of claim 1, wherein the CSA can function as the local HA in one of: (i) an exclusive state in which the HA in the host does not track the dataset; or (ii) a shared state in which both the CSA and the HA track the dataset and an RA in the host has a cached copy of the dataset.

7. The HA is receiving a request from an RA to modify the data set after the CSA begins functioning as the local HA; 2. The hardware accelerator of claim 1, further configured to: send a request to the CSA to transition to a disabled state in which the CSA no longer functions as the local HA, and ownership of the data set is returned to the HA.

8. 1. An I / O device configured to be communicatively coupled to a host, the I / O device comprising: Accelerator function and a CSA communicatively coupled to the accelerator function and to a HA within the host, the CSA and the HA being part of the same coherent domain; the CSA is configured to request ownership of the data set from the HA such that the CSA acts as a local HA for the data set; The accelerator function indicates whether the CSA tracks the data set at a cache line level or a data block level when functioning as the local HA.

9. 9. The I / O device of claim 8, wherein before the CSA requests ownership of the data set from the HA, the accelerator function is configured to send a request to the CSA to perform at least one of reading or modifying the data set, the request from the accelerator function indicating whether the CSA is to track the data set at the cache line level or the data block level.

10. 10. The I / O device of claim 9, wherein the HA is configured to respond to the request from the CSA by snooping to an RA in the host to flush a cached copy of the dataset from the host, the RA being capable of storing a cached copy of the dataset.

11. 11. The I / O device of claim 10, wherein the HA is configured to send a push CMO to the CSA after receiving an indication that the snoop is complete, the push CMO causing any cached copies of the data set stacked in a buffer or last level cache to be moved to a slave agent within the CSA.

12. 12. The I / O device of claim 11, wherein after determining that the slave agent has received the push CMO, the CSA transitions to act as a local HA for the data set.

13. 9. The I / O device of claim 8, wherein the CSA can function as the local HA in one of (i) an exclusive state in which the HA in the host does not track the dataset, or (ii) a shared state in which both the CSA and the HA track the dataset and an RA in the host has a cached copy of the dataset.

14. 1. A method comprising: communicatively coupling a host including an RA and an HA to an I / O device, the I / O device including a CSA communicatively coupled to an accelerator function within the I / O device and the HA, the CSA and the HA being part of a same coherent domain; using the CSA to request ownership of the data set from the HA so that the CSA acts as a local HA for the data set; The method, wherein the accelerator function indicates whether the CSA tracks the data set at a cache line level or a data block level when acting as the local HA.

Images