Embedded Configurable Engine

Abstract

Embodiments herein describe a configurable engine integrated into a processor's cache hierarchy. The configurable engine can enable efficient data sharing between main memory, cache memory, and cores. The configurable engine can perform more efficient operations within the cache hierarchy. In one embodiment, the configurable engine is controlled (or configured) by software (e.g., an operating system (OS)) and is tailored to each application domain. That is, the OS can configure the engine according to the data flow profile of a particular application being executed by the processor.

Description

[Technical Field]

[0001] Embodiments of the present disclosure generally relate to incorporating a configurable engine into a processor to manage data flowing between caches, main memory, and cores within the processor. [Background technology]

[0002] Modern central processing units (CPUs) suffer from significant inefficiencies due to mispredictions and data movement inefficiencies in the cache hierarchy. Applications with irregular workloads can generate irregular data flow patterns when storing and retrieving data from caches and main memory. If the processor provides efficient, customized implementations for these irregular workloads, these applications can run faster. Currently proposed solutions to these inefficiencies include placing dedicated execution engines in the main data path inside the processor's cores. This form of on-core integration provides the most fine-grained integration possible, as the engines operate in a similar manner to other execution units. However, this form of integration introduces several undesirable challenges in practical implementations. First, the engines are inserted in areas that significantly impact core performance. As a result, optimizing core performance becomes more complex. Second, such integration prevents the execution of some operations that would be more beneficial if performed closer to memory. Furthermore, CPU cores operate at very high frequencies, which has a wide gap with the typical performance of field programmable gate arrays (FPGAs) and other data flow engines that may be inserted into the cores.

[0003] As an example, the instruction dispatcher is typically responsible for scheduling operations on the engine in addition to operations on other execution units. It must track instruction dependencies through register renaming, among other complex tasks. As a result, the dispatcher becomes more complex due to its integration on the execution engine core.

[0004] Specialized instructions (which map to operations on the engine) can be delivered to the dispatcher in the same way as other instructions. Because a small number of operands are allowed, the set of operations on the execution engine is limited. Some instruction set architecture (ISA) extensions are provided for single instruction, multiple data (SIMD) operations. However, ISA extensions complicate the core design and can degrade core performance. Backward compatibility concerns are another barrier to ISA extensions. Therefore, ISA extensions are not feasible for many applications, especially those with irregular workloads. Summary of the Invention [Means for solving the problem]

[0005] Techniques are described for a processor. One embodiment is a processor including a first core with a first private cache at a first level of a cache hierarchy, a second core with a second private cache at the first level of the cache hierarchy, and a shared cache at a second level of the cache hierarchy, the shared cache comprising a configurable engine, the configurable engine comprising circuitry configured to manage data transferred between the first and second private caches and the shared cache. Further, the configurable engine is reconfigurable by software to perform tasks for different user applications executing on the first and second cores.

[0006] Another embodiment described herein is a method that includes providing a configurable engine in a shared cache of a processor having multiple cores with private caches, the configurable engine having circuitry for managing data transferred between the private cache and the shared cache; receiving a library or framework for the configurable engine; providing runtime information from a user application to the configurable engine using the library or framework; and linking the library or framework to the user application.

[0007] BRIEF DESCRIPTION OF THE DRAWINGS In a manner in which the above-recited features may be understood in detail, a more particular description briefly summarized above can be had by reference to exemplary embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical exemplary embodiments and therefore should not be considered as limiting the scope thereof. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an embedded configurable engine within a processor's cache hierarchy, according to one embodiment. [Figure 2A] Each illustrates a distributed, embedded, configurable engine within a processor's cache hierarchy, according to one embodiment. [Figure 2B] Each illustrates a distributed, embedded, configurable engine within a processor's cache hierarchy, according to one embodiment. [Figure 3] FIG. 2 is a logical diagram of an embedded configurable engine within a cache hierarchy, according to one embodiment. [Figure 4] 1 illustrates an embedded configurable engine, according to one embodiment. [Figure 5] 1 is a workflow for configuring an embedded configurable engine, according to one embodiment. [Figure 6]10 is a flowchart for configuring an embedded configurable engine for use in a user application, according to one embodiment. [Figure 7] 1 is software code for configuring an embedded configurable engine for use in a user application, according to one embodiment. [Figure 8] 1 illustrates a scenario in which several applications communicate with an embedded configurable engine using a shared library, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] For ease of understanding, wherever possible, identical reference numbers have been used to indicate identical elements common to the figures. It is contemplated that elements of one embodiment may be beneficially incorporated in other embodiments.

[0010] Various features are described below with reference to the drawings. It should be noted that the drawings may or may not be drawn to scale, and that elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should be noted that the drawings are intended only to facilitate the description of features. They are not intended as an exhaustive description of the specification or as limitations on the scope of the claims. In addition, the illustrated example need not have all the aspects or advantages shown. An aspect or advantage described in connection with a particular embodiment is not necessarily limited to that embodiment and may be implemented in any other embodiment, even if not so illustrated or explicitly described.

[0011] Embodiments herein describe an embedded configurable engine that can address the above-mentioned CPU inefficiencies without requiring the complexity of ISA extensions. In one embodiment, the configurable engine is embedded in the processor's cache hierarchy rather than in the processor's core. Nevertheless, the configurable engine can enable efficient data sharing between main memory, cache memory, and the core. Furthermore, the engine can be a dataflow engine that is configurable, allowing customization for various target domains with the same or similar architecture. This is particularly useful for applications with irregular workloads.

[0012] In one embodiment, the configurable engine is controlled (or configured) by software, such as an operating system (OS), and is tailored to each application domain. That is, the software can configure the engine according to the data flow profile of the particular application being executed by the processor. Furthermore, the software can dynamically reconfigure the configurable engine through firmware updates with minimal disruption to existing flows.

[0013] 1 illustrates an embedded configurable engine 150 within the cache hierarchy of a processor 100, according to one embodiment. Processor 100 may be an integrated circuit (or chip) that includes multiple cores 105. While FIG. 1 illustrates four cores 105A-D, processor 100 may include any number of cores.

[0014] In this example, each core 105 includes an L1 cache 110 and an L2 cache 115. In one embodiment, these caches 110, 115 are private (or non-shared) caches that are not accessible to other cores 105 in the processor 100. In contrast, the processor also includes an L3 cache 120 that is shared by the cores 105. That is, any of the cores 105 can store and read data from the L3 cache 120.

[0015] In this example, configurable engine 150 is physically located in a portion of processor 100 that includes L3 cache 120. In one embodiment, configurable engine 150 facilitates communication between L3 cache 120 (and main memory (not shown)) and L2 caches 115A-D in cores 105A-D. For example, engine 150 may be used in conjunction with a prefetcher implemented within engine 150 to retrieve data from L3 cache 120 (or main memory) and store the data in L2 cache 115 or L1 cache 110 in core 105. For example, each core 105 may include a respective prefetcher.

[0016] In a traditional prefetcher model, the prefetcher learns and predicts data request patterns. However, in a domain-specific model, the prefetcher knows the layout and usage of the data. For example, in a software-influenced paradigm, data movement can be abstracted from the domain-specific model so that the prefetcher knows the data layout and usage without having to learn or predict patterns. Configurable engine 150 provides a method for using a domain-specific prefetcher to prefetch data from L3 cache 120 (or main memory) and store the data in L2 cache 115 of a particular core 105. However, prefetching is just one use of configurable engine 150. For example, in another embodiment, configurable engine 150 can be used as a data compression engine. In another example, configurable engine 150 can observe data being transferred between the L2 and L3 caches. For example, to detect side-channel attacks, engine 150 can observe the state of different data as it is transferred between caches. In another embodiment, the engine affects programs running on the cores by observing data being transferred and modifying shared data. In another embodiment, the configurable engine 150 can be configured to perform common operations for the system.

[0017] In one embodiment, configurable engine 150 includes programmable logic circuitry (e.g., a lookup table, a configurable logic block (CLB), etc.). In one embodiment, configurable engine 150 may be an FPGA or a coarse-grained reconfigurable architecture (CGRA) embedded in an integrated circuit of processor 100. In another embodiment, configurable engine 150 is formed using hardened circuitry, such as circuitry used to implement a dataflow engine. In that case, configurable engine 150 may be less flexible than when formed using an FPGA or CGRA, but may still be configurable to perform several different functions (e.g., prefetching or data compression) or to enable several different dataflow patterns. For example, configurable engine 150 may be a dataflow engine, an FPGA fabric, or a bus-configured FPGA fabric.

[0018] 1 illustrates the incorporation of configurable engine 150 into L3 cache 120, but this is by way of example only. In one embodiment, configurable engine 150 may be incorporated into any shared or non-shared cache level. For example, processor 100 may have an L4 shared cache, in which case configurable engine 150 may be incorporated into that cache. In one embodiment, configurable engine 150 is incorporated into the highest cache memory in the cache hierarchy, immediately below main memory (e.g., RAM).

[0019] FIG. 2A illustrates an embedded configurable engine 150 distributed within the cache hierarchy of a processor 200, according to one embodiment. Similar to processor 100 of FIG. 1, processor 200 is an integrated circuit (or chip) that includes multiple cores 105. While FIG. 2A illustrates four cores 105A-D, processor 200 may include any number of cores. Core 105 of FIG. 2A also includes a private L1 cache 110 and a private L2 cache 115, as shown in FIG. 1.

[0020] However, instead of the configurable engine being located in a contiguous portion of L3 cache 120, in this embodiment the configurable engine is distributed throughout L3 cache 120. As shown, the configurable engine is divided into four configurable engine portions 250A-D that are disposed within L3 cache 120 proximate to four cores 105A-D. That is, configurable engine portion 250A is disposed in an area of ​​L3 cache 120 proximate to core 105A, configurable engine portion 250B is disposed in an area of ​​L3 cache 120 proximate to core 105B, configurable engine portion 250C is disposed in an area of ​​L3 cache 120 proximate to core 105C, and configurable engine portion 250D is disposed in an area of ​​L3 cache 120 proximate to core 105D. These separate portions 250 may be thought of as a single distributed configurable engine or as multiple separate configurable engines.

[0021] In some implementations, there may be performance advantages to having the circuitry of a configurable engine located in contiguous locations within L3 cache 120, as shown in Figure 1. However, in other implementations, there may be performance advantages to distributing the circuitry within a configurable engine such that portions 250 are located closer to each core 105 (i.e., portions 250 are non-contiguous), as shown in Figure 2A.

[0022] 2A illustrates distributing configurable engine portion 250 within L3 cache 120, but this is by way of example only. In one embodiment, configurable engine 150 may be incorporated and distributed within any shared cache level. In one embodiment, configurable engine portion 250 is incorporated and distributed within the highest cache memory immediately below main memory (e.g., RAM) in the cache hierarchy.

[0023] In another embodiment, both the distributed configurable engine and the common configurable engine operate in concert as shown in FIG. 2B. Each portion of the distributed configurable engines 250A-D observes memory accesses to / from its associated cores 105A-D for some or all address ranges. Each portion of the distributed engines 250A-D can notify the common configurable engine 260 of the observed accesses or forward access requests to the engine 260. The common configurable engine 260 uses the information according to predetermined operations for the current workload. The common configurable engine 260 can interface to the cache hierarchy through cache 270. The cache 270 is located outside of any core but behaves like a core's L2 cache, e.g., caches 115A-D. Such a configuration allows the engines to be integrated using existing cache coherency mechanisms.

[0024] Figure 3 is a logical diagram of an embedded configurable engine 150 within a cache hierarchy, according to one embodiment. This logical diagram is the same whether the configurable engine is disposed in a contiguous region of the L3 cache, as shown in Figure 1, or whether the configurable engine is distributed across different non-contiguous regions of the L3 cache, as shown in Figure 2A or 2B.

[0025] 3, the core includes an execution unit 305 that sends data to and receives data from an L1 cache 110 and an L2 cache 115. In this embodiment, these caches 110, 115 are private caches that can only be accessed by the execution units within the core 105 (and not by other cores within the processor).

[0026] Configurable engine 150 is logically disposed between L2 cache 115 and L3 cache 120. That is, configurable engine 150 manages the flow of data between L2 cache 115 and L3 cache 120. Furthermore, FIG. 3 illustrates that configurable engine 150 can exchange both control data and application data. Application data can include data used (and generated) by execution unit 305 when executing a particular user application. Control data exchanged between engine 150 and L2 cache 115 allows engine 150 to query the state of cache 115 and invalidate particular cache contents. Control data exchanged between engine 150 and L3 cache 120 can perform similar functions. Furthermore, FIG. 3 illustrates L3 cache 120 in communication with main memory 310 (e.g., RAM).

[0027] FIG. 3 also illustrates configurable engine 150 receiving configuration data from OS 315. The configuration data allows configurable engine 150 to be configured (or programmed) to perform dataflow tasks. In one embodiment, OS 315 configures engine 150 to perform customized dataflow tasks that correspond to a dataflow pattern for a particular user application being executed by execution unit 305. For example, the user application may have an irregular workload that corresponds to an irregular dataflow pattern. Instead of requesting an in-core prefetcher to attempt to learn this pattern, OS 315 can configure engine 150 to perform dataflow tasks for this pattern. In this software-centric example, because OS 315 has knowledge of the irregular dataflow pattern, it can configure engine 150 to perform the customized dataflow task for the dataflow pattern. Configuring configurable engine 150 using OS 315 is described in more detail below in FIG. 5. Although embodiments herein describe using OS 315 to configure engine 150, other types of software can be used to configure engine 150 in a manner that is transparent to the user.

[0028] FIG. 3 illustrates an embodiment in which engine 150 resides between private L2 cache 110 and shared L3 cache 120. Engine 150 can observe activity of the attached cache, similar to a hardware prefetcher. For example, engine 150 is aware of memory accesses and whether the accesses result in a miss or a hit. Engine 150 can observe and perform many activities like a cache controller. For example, engine 150 can view and update cache line flags and determine whether a cache line is filled or evicted. Additionally, engine 150 can perform the following operations on the connection between the L2 and L3 caches: a) intercept, block, and / or insert accesses to the L3 cache; and b) intercept, block, and / or insert responses to the L2 cache. These operations enable configurable engine 150 to manage data movement within the cache hierarchy, both in and out of core 105. The engine can act on the data and return the resulting data to the cache hierarchy. Engine 150 can be configured to ignore certain address ranges and not recognize memory accesses within those ranges or memory accesses of specific applications. In effect, engine 150 can interact at a granular level with tasks running on core 105 and define the context of those tasks. Notably, this capability exceeds the capabilities of hardware prefetchers or typical L1 / L2 cache controllers.

[0029] FIG. 4 illustrates an embedded configurable engine 150 according to one embodiment. The engine 150 includes two types of components: common components (e.g., shared circuitry 405) and per-core components (e.g., core-specific circuitry 410). The shared circuitry 405 is shared between the cores in the processor and other components associated with each core. Multiple engines 150 may exist within a single integrated circuit or die. Engines on different dies may cooperate, for example, when an OS migrates threads from one die to another. The shared circuitry 405 may implement multiple engines and assign them to the cores that use them. Additionally, the shared circuitry 405 allows tracking of activity across multiple cores.

[0030] A per-core component (e.g., core-specific circuitry 410) may provide latency-critical services and implement a configurable engine dedicated to the attached core. For example, core-specific circuitry 410A may be dedicated to one core, while core-specific circuitry 410N may be dedicated to another core. Although the configurable engine is shown in FIG. 4 as a contiguous region (e.g., as shown in FIG. 1) including shared circuitry 405 and core-specific circuitry 410, this functionality may be physically distributed across multiple portions (or separate engines) across a cache hierarchy (e.g., as shown in FIGS. 2A and 2B).

[0031] In one embodiment, the engine has access to a Table Look-aside Buffer (TLB) 415 for mapping virtual addresses to physical addresses (or performing address translation), and to a reverse TLB for mapping physical addresses to virtual addresses. Although operations may involve virtual addresses, these mappings are advantageous because memory accesses in the L2 or L3 cache use physical addresses. Thus, the inputs and outputs of engine 150 may be either physical or virtual addresses.

[0032] Alternatively, two sets of content addressable memory (CAM) managed directly by the OS can be used for address translation. This has the advantage that the processor's address lookup mechanism does not need to be modified to take the new buffer into account. In addition, it may be beneficial to allow engine 150 to access memory for a task's intermediate data. In one embodiment, engine 150 can read the intermediate data from memory and write it to an internal scratchpad memory.

[0033] 5 illustrates a workflow for configuring an embedded configurable engine, according to one embodiment. In one embodiment, an instruction scheduler within each core may be responsible for scheduling tasks running on the engine within the core. However, because the configurable engine described above may be integrated into a cache hierarchy, scheduling of engine tasks may be done differently.

[0034] In one embodiment, dispatching a task to a configurable engine in the cache hierarchy occurs in two steps. First, the OS (or other type of software) initializes the task on the configurable engine based on information from the application. Then, target activity in the attached cache wakes up the task to perform the specified operation. In one embodiment, there are two types of activity. One relates to memory accesses flowing through the engine from either the L2 or L3 cache. The accesses can be of different types, and the engine can be programmed to act on them based on the type of access (e.g., prefetch access, write-through, write-back, table walk access) or target address, i.e., to take different actions depending on which range the address belongs to. The other type of activity relates to activity observed by the engine 150 through the control channel by exchanging control data. For example, if there is a cache miss / hit / fill in L2 / L3 on a specific address range (configured via the OS).

[0035] A high-level interaction between a configurable engine and an OS is shown in FIG. 5. When the OS loads the binary of an application that uses a configurable engine, the OS also configures the configurable engine using the bitstream, as indicated by arrow 505. In one embodiment, the OS first checks whether there is enough room in the configurable engine to perform the task before configuring the engine. The application can wait until an engine is available or instruct the OS to run without an engine. The configurable engine can wait for runtime information from the application and for a task to become active. That is, after the configurable engine is configured, it can be idle, waiting for runtime information and for a task to become active.

[0036] As indicated by arrow 510, an application provides runtime information to engine 150, bringing engine 150 into a ready state. In one embodiment, an application initiates the flow of runtime information to engine 150, but the OS is responsible for relaying the runtime information to engine 150. For example, an application may not be able to interact with the engine directly.

[0037] In one embodiment, the engine may receive multiple pieces of information throughout the application's lifetime. For example, an application may include several phases. Therefore, the optimal prefetching strategy may depend on these phases. For example, a graph application may process fan-out nodes early on, but process fan-in nodes later. The two phases may cause the configurable engine to monitor different address ranges. In this case, the application must at least instruct the engine to switch behavior. In some cases, a task may become active without such information.

[0038] Once a task is activated (e.g., configuration engine 150 is placed in an active state), engine 150 monitors and performs the task when predetermined conditions are met. That is, engine 150 manages data being transferred between caches, which may include observing the data, moving the data, compressing the data, processing the data, or acting on the data.

[0039] The trigger condition may rely on an additional bit that the OS sets based on application behavior and propagates through the page table entry to the metadata of each memory access. The bit value is directly associated with the operation or class of operations that is selected by the engine based on other runtime conditions of the system. The condition may also include a set of address ranges associated with the operation that is sent to the engine through other means.

[0040] Using the workflow of Figure 5, the OS can allocate, schedule, and load tasks within engine 150. Additionally, the OS can repeat Figure 5, configuring engine 150 to perform tasks for a first user application (being executed by a core) during a first period, and then reconfiguring engine 150 to perform tasks for a second user application during a second period. An engine can perform several tasks for different applications simultaneously.

[0041] Additionally, the OS can perform load balancing across engines 150 and cores to provide better overall system performance. For example, if there is no room in the engine and the new application has a low priority, the OS can choose to use an engine in a core rather than a configurable engine 150 in the shared cache. In that case, the OS will not configure engine 150 to perform the task. However, if the new application has a high priority, the OS can replace the current implementation of engine 150 performing tasks for the application with a lower priority with a new implementation of engine 150 (using the workflow of FIG. 5) to perform tasks for the new, higher priority application.

[0042] 6 is a flowchart for configuring an embedded configurable engine for use by a user application, according to one embodiment. In one embodiment, a software programmer is aware of the capabilities of the configurable engine and provides implementations of operations that benefit the application along with the normal software implementation. The implementations can be obtained from a catalog or built specifically for the application.

[0043] In block 605, a software developer receives or creates libraries for an application that uses the configurable engine. An application may include different phases that use different sets of operations. There may not be enough room in the engine to perform a requested operation. In that case, the OS can fall back to an on-core implementation. Alternatively, if the compiled application uses some functionality from dynamically linked libraries, these libraries can augment other implementations on the engine. When updated libraries are installed on a system with the configurable engine, the application can be serviced by the engine without recompilation. This is possible because of the nature of dynamic linking.

[0044] At block 610, the application calls an application programming interface (API) to communicate runtime information to the configurable engine. This is shown below in FIG. 7 below at label "2." In one embodiment, the library provides APIs to further configure the engine to perform different operations during application operation. Generally, the library allows user applications to interact with the configurable engine, for example, using one or more APIs.

[0045] In one embodiment, instead of receiving a library, an application can use a framework to communicate runtime information to the configurable engine, in which case the application may not need to use an API to communicate with the configurable engine.

[0046] At block 615, the compiler links the library to the application. In one embodiment, multiple applications (e.g., different user applications) within a domain are linked to a shared library. In one embodiment, the library encapsulates a mechanism for communicating with a configurable engine (e.g., that can act as a prefetcher) through the OS kernel using model-specific registers (MSRs) or some other method.

[0047] Figure 7 illustrates software code 700 for enabling interaction with a configurable engine embedded in a cache hierarchy using a library 705. Figure 7 illustrates an application written to use a domain-specific prefetcher for the graph analysis domain. The application includes a library header, calls a specific API (e.g., block 610 in Figure 6), and links with the library (block 615 in Figure 6). The library header is labeled "1" in Figure 7, the call to the API is labeled "2" in Figure 7, and the instruction to the compiler to link with the library is labeled "3" in Figure 7.

[0048] Figure 8 illustrates a scenario in which several applications (e.g., different user applications) in the graph analytics domain can communicate with an embedded configurable engine using a shared library. The library encapsulates an OS kernel component that has the authority to control and monitor the engine through one or more MSRs. Other communication modes can use control registers (CRs), shared addresses, or configuration bits. Libraries from different domains can use common components (e.g., shared circuit 405 in Figure 4) to interact with the engine.

[0049] The configurable engine described above can perform general-purpose operations useful for most applications, such as a hardware prefetcher or data compression. In such cases, configuration of the engine can be performed by firmware at system boot time, even before the OS is online. Updated versions of the operations can be delivered through firmware updates. Systems with the same configurable engine may have different firmware. System operators can update the firmware appropriate for the current use case. For example, a system used primarily as a search engine may use one firmware version for the configurable engine, while another system used as a database system may use a different firmware version for its configurable engine.

[0050] Embodiments herein enable the acceleration of tightly coupled tasks within a core because they share the same data path, i.e., cache hierarchy. Meanwhile, embodiments herein may not disrupt an already challenging core design. Different devices using the same core design for different use cases can include different variations of the configurable engine without modifying the core itself. When the engine is located within the core itself, it is initiated and controlled through special instructions in the ISA. Thus, the benefits of such an engine are limited to operations compatible with the ISA limitations, i.e., the maximum number of operands. In contrast, applications can interact with a configurable engine embedded in the cache hierarchy through the OS and libraries. Thus, any application can benefit from providing preferred operations without affecting other applications by modifying the ISA. As a result, embodiments herein describe an easy-to-implement technique for providing vendor-specific differentiating features. Domain-specific frameworks can be used to further simplify user interaction. Such frameworks encapsulate the interaction with the engine so that users may not need to use the API directly.

[0051] Additionally, embodiments herein may enable tasks to operate closer to memory to reduce latency and / or energy. The engine can observe and act on data without (or before) propagating the data to the core. As an example, this latency advantage allows the engine to implement a prefetcher for the graph analysis domain, improving application performance by up to 6%. Furthermore, nearly half of a core's pipeline cycles are stalled by data and instruction fetches. Embodiments herein can reduce the amount of instructions and data fetched to the core, meaning that embodiments herein not only enable the efficient implementation of larger operations compared to existing solutions, but also help improve overall system performance.

[0052] Some systems deploy configurable engines away from cores via an interconnect, such as a Network of Computing (NoC), either on-chip or off-chip. Therefore, latency between the engine and cores is high. Furthermore, frequent communication between the engine and cores uses memory bandwidth and degrades overall system performance. As a result, such integration can only provide acceleration for tasks that require little communication with the core. A known use case is for tasks in which the core configures the engine and the input and output locations of an operation. The engine's input or output may be directly connected to input / output (IO) circuitry. The core is notified when the operation is complete. These embodiments herein propose a method and system structure that enables the acceleration of tasks more tightly coupled to the core. In particular, this structure allows doing so without adversely affecting the core design. This allows for operation closer to main memory per domain requirements while taking advantage of the locality provided by cache hierarchies. The impact is to address the inefficiencies of data movement outside of the existing load / store paradigm.

[0053] Some other solutions include a programmable prefetcher unit (PPU) for accelerating applications with irregular workloads. However, the PPU cannot control and set the context of the tasks running on the core. The PPU can only help deliver data faster. In contrast, using embodiments herein, the OS can store read-only data in the engine. When a core makes a read access to the data, the engine can provide the data and block the access from propagating, saving memory bandwidth. However, the PPU must retrieve the data. Furthermore, the PPU is located inside the core itself, not in a shared cache memory. Furthermore, the PPU is off the instruction fetch path and cannot operate on the instruction stream. Furthermore, while the PPU uses a traditional processor architecture, embodiments herein propose a dataflow engine that may be more power-efficient for the read-modify-write operations of a typical workload.

[0054] Some other solutions employ engines for the memory services programming model, where the goal of the system is to bring computation closer to the data. Consequently, the engines are small and distributed, increasing the overhead of configurability and making the implementation less practical. In contrast, embodiments herein employ larger engines and may address the data movement inefficiencies of cores by configuring the engines for the domains running on the cores. Furthermore, the method of using configurable engines differs from engines for the memory services programming model.

[0055] Some other solutions use generic engines to provide data movement optimizations. These generic engines allow software to modify cache behavior using callbacks, which are short tasks created and scheduled entirely by hardware within the engine. In contrast, the engines described herein can use the OS to allocate, schedule, and load tasks. Thus, the OS can perform load balancing across engines in addition to cores to provide better overall system performance. For example, if there is no room in the engine and a new application has a low priority, the OS can choose to implement the task on a core rather than a configurable engine in the shared cache. However, if the priority is high, the OS can replace the implementation of the operation of an application with a lower priority with its corresponding core implementation to make room for the new application. Furthermore, generic engines may rely on phantom addresses (virtual addresses that are not mapped to physical addresses and only exist in the cache) to start the engine. Instead, in one embodiment, the configurable engine described herein uses a hardware-only solution, does not rely on phantom addresses, and can avoid OS modifications to support them.

[0056] In the foregoing, reference is made to the embodiments presented in this disclosure. However, the scope of the disclosure is not limited to the specific described embodiments. Instead, any combination of the described features and elements, whether associated with different embodiments or not, is contemplated for implementing and practicing the contemplated embodiments. Moreover, while the embodiments disclosed herein may achieve advantages over other possible solutions or prior art, whether or not a particular advantage is achieved by a given embodiment does not limit the scope of the disclosure. Accordingly, the foregoing aspects, features, embodiments, and advantages are merely illustrative and are not considered elements or limitations of the appended claims unless expressly recited in the claims.

[0057] As will be appreciated by one skilled in the art, embodiments disclosed herein may be embodied as a system, method, or computer program product. Accordingly, aspects may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be generally referred to herein as a "circuit," "module," or "system." Furthermore, aspects may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied therein.

[0058] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this specification, a computer-readable storage medium is any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0059] A computer-readable signal medium may include a propagated data signal in which computer-readable program code is embodied, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium but may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0060] The program code embodied on the computer readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

[0061] Computer program code for carrying out operations of aspects of the present disclosure may be written in any combination of one or more programming languages, including, for example, object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0062] Aspects of the present disclosure are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments presented in this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in the flowchart and / or block diagram blocks.

[0063] These computer program instructions may also be stored on a computer-readable storage medium, and the instructions may direct a computer, programmable data processing apparatus, and / or other device to function in a particular manner to produce an article of manufacture including instructions that perform the functions / acts specified in the flowchart and / or block diagram blocks.

[0064] Computer program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to create a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide a process for performing the functions / acts specified in the flowchart and / or block diagram blocks.

[0065] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0066] While the above is directed to particular examples, other and further examples may be devised without departing from the basic scope thereof, which scope is determined by the following claims.

Claims

1. 1. A processor, comprising: a first core having a first private cache at a first level of a cache hierarchy; a second core comprising a second private cache at the first level of the cache hierarchy; a shared cache at a second level of the cache hierarchy, the shared cache comprising a configurable engine, the configurable engine comprising circuitry configured to manage data transferred between the first and second private caches and the shared cache; A processor wherein the configurable engine is reconfigurable by software to perform tasks for different user applications running on the first and second cores.

2. the software being an operating system (OS), and reconfiguring the configurable engine using the OS; 2. The processor of claim 1, further comprising configuring the configurable engine using a bitstream provided by the OS when the OS loads a binary of one of the different user applications.

3. reconfiguring the configurable engine using the OS after configuring the configurable engine; maintaining the configurable engine in an idle state until receiving runtime information from the one of the different user applications; providing the runtime information to the configurable engine to bring the configurable engine into a ready state; activating the configurable engine to perform the task when a predetermined condition is met.

4. 2. The processor of claim 1, wherein the configurable engine is configured to intercept, block, and insert accesses to at least one of: (i) the first and second private caches; or (ii) the shared cache.

5. The processor of claim 1 , wherein the configurable engine is disposed in a contiguous portion of the shared cache.

6. 2. The processor of claim 1, wherein the configurable engine comprises a plurality of portions distributed across non-contiguous regions within the shared cache, a first portion of the portions being disposed proximate to the first core and a second portion of the portions being disposed proximate to the second core.

7. 7. The processor of claim 6, wherein the configurable engine comprises a common configurable engine, and wherein the portions of the configurable engine notify the common configurable engine of accesses observed at their respective cores or forward access requests to the common configurable engine.

8. 2. The processor of claim 1, wherein the configurable engine is configured to access memory and translate addresses between physical and virtual memory addresses when transmitting data between the first and second private caches and the shared cache.

9. 2. The processor of claim 1, wherein the configurable engine is configured to read intermediate data from memory and write it to an internal scratchpad memory, and to write final results to the memory.

10. 2. The processor of claim 1, wherein the software is an operating system, the operating system configured to load balance the configurable engines based on the priorities of the different user applications.

11. The processor of claim 1 , wherein the configurable engine is one of a dataflow engine, an FPGA fabric, or a bus-configured FPGA fabric.

12. 1. A method comprising: providing a configurable engine in a shared cache of a processor having multiple cores with private caches, the configurable engine comprising circuitry for managing data transferred between the private caches and the shared cache; receiving a library or framework for the configurable engine; providing runtime information from a user application to the configurable engine using the library or framework; linking the library or framework to the user application.

13. The library allows the user application to interact with the configurable engine, and the method comprises: The method of claim 12 , further comprising calling an application programming interface (API) provided by the library to provide the runtime information to the configurable engine.

14. 13. The method of claim 12, further comprising linking the library to a plurality of applications that allows the applications to communicate with the configurable engine through an OS kernel.

15. The method of claim 12 , further comprising using the configurable engine on behalf of the user application to prefetch data or perform data compression.

16. The method of claim 12 , wherein the configurable engine is disposed in a contiguous portion of the shared cache.

17. 13. The method of claim 12, wherein the configurable engine comprises a plurality of portions distributed across non-contiguous regions within the shared cache, a first portion of the portions being disposed proximate to a first core within the processor, and a second portion of the portions being disposed proximate to a second core within the processor.

18. 20. The method of claim 17, wherein the configurable engine comprises a common configurable engine, and wherein the multiple portions of the configurable engine notify the common configurable engine of accesses observed at their respective cores or forward access requests to the common configurable engine.

19. 13. The method of claim 12, further comprising accessing memory with the configurable engine to translate addresses between physical and virtual memory addresses when transmitting data between the private cache and the shared cache.

20. The method of claim 12 , further comprising load balancing the configurable engines based on priorities of different user applications.

Images