Configurable engine embedded in processor for managing cached data streams

By embedding configurable engines in the processor's cache level, the problem of inefficiency of CPU is solved, efficient data processing and performance improvements for irregular workload applications are achieved, and complexity brought about by ISA expansion is avoided.

CN119072683APending Publication Date: 2024-12-03XILINX INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202480002286.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-02-21
Filing Date
2024-02-16
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Modern CPUs are inefficient due to mispredictions and inefficiencies in data movement in the cache hierarchy, especially for irregular workloads. Existing solutions such as dedicated execution engine integration into the processor core will lead to performance complexity. and compatibility issues.

Method used

Embed a configurable engine in the cache level of the processor, and manages the transfer of data between main memory, cache and core through software configuration, avoids the complexity of ISA expansion, and realizes customized data stream processing for different applications .

Benefits of technology

It realizes efficient data sharing and processing, improves the overall performance of the CPU, especially for irregular workload applications, reduces delay and energy consumption, and enhances the flexibility and compatibility of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072683A_ABST
    Figure CN119072683A_ABST
Patent Text Reader

Abstract

Embodiments herein describe a configurable engine that is embedded in a cache hierarchy of a processor. The configurable engine may enable efficient data sharing among the main memory, the cache memory, and the core. The configurable engine may perform operations that are more efficiently completed in the cache hierarchy. In one embodiment, the configurable engine is controlled (or configured) by software, such as an operating system (OS), adapted to each application domain. That is, the OS may configure the engine according to a data flow profile of a particular application being executed by the processor.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Examples of the present disclosure generally relate to embedding a configurable engine into a processor to manage data flowing between caches, main memory, and cores in the processor. Background Art

[0002] Modern central processing units (CPUs) suffer from significant inefficiencies due to mispredictions and inefficient data movement in the cache hierarchy. Applications with irregular workloads may generate irregular data flow patterns when storing and retrieving data from cache and main memory. If the processor provides efficient custom implementation for these irregular workloads, these applications can be executed faster. Current solutions proposed for these inefficiencies include placing dedicated execution engines in the main data path within the core of the processor. This form of core integration provides the finest possible integration because the engine performs operations in a manner similar to other execution units. However, this form of integration can bring several unexpected challenges to actual implementation. First, the engine is inserted in an area that has a large impact on core performance. As a result, performance optimization of the core becomes more complicated. Second, this type of integration does not allow the execution of some operations that would be more beneficial if performed closer to the memory. In addition, the CPU core runs at a very high frequency, which is far from the typical performance of field programmable gate arrays (FPGAs) and other data flow engines that can be inserted into the core.

[0003] As an example, the instruction scheduler is typically responsible for scheduling operations on the engine in addition to operations of other execution units. It must keep track of instruction dependencies through register renaming, among other complex tasks. As a result, the scheduler becomes more complex as the execution engine is integrated on the core.

[0004] Special instructions (operations mapped to the engine) can be delivered to the scheduler in the same manner as other instructions. Because a small amount of operands are allowed, a set of operations on the execution engine is limited. Some instruction set architectures (ISA) are provided for single instruction multiple data (SIMD) operations. However, ISA expansion complicates the core design and may degrade core performance. The backward compatibility problem is another obstacle to ISA expansion. Therefore, ISA expansion is infeasible for many applications, especially those with irregular workloads. Summary of the invention

[0005] Techniques for a processor are described. One example is a processor that includes a first core, a second core, a shared cache, the first core including a first private cache in a first level of a cache hierarchy, the second core including a second private cache in the first level of the cache hierarchy, the shared cache including a configurable engine including circuitry configured to manage data passed between the first private cache and the second private cache and the shared cache. In addition, the configurable engine can be reconfigured by software to perform tasks for different user applications executing on the first core and the second core.

[0006] Another embodiment described herein is a method comprising: providing a configurable engine in a shared cache of a processor, wherein the processor includes multiple cores with private caches, wherein the configurable engine includes circuitry to manage data passed between the private cache and the shared cache; receiving a library or framework for the configurable engine; using the library or framework to provide runtime information to the configurable engine from a user application; and linking the library or framework to the user application. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to be able to understand in detail the manner in which the above features are described, a more specific description briefly summarized above may be obtained by reference to example implementations, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical example implementations and therefore should not be considered to limit the scope thereof.

[0008] Figure 1 An embedded configurable engine in a cache hierarchy of a processor according to an example is illustrated.

[0009] Figure 2A and Figure 2B Each illustrates a distributed embedded configurable engine in a cache hierarchy of a processor according to an example.

[0010] Figure 3 is a logical view of an embedded configurable engine in a cache hierarchy according to an example.

[0011] Figure 4 An embedded configurable engine according to an example is illustrated.

[0012] Figure 5 is a workflow for configuring an embedded configurable engine according to an example.

[0013] Figure 6 is a flow chart for configuring an embedded configurable engine for use by a user application according to an example.

[0014] Figure 7 is software code for configuring an embedded configurable engine for use by a user application according to an example.

[0015] Figure 8 A situation in which several applications use a shared library to communicate with an embedded configurable engine according to an example is illustrated.

[0016] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements of one example may be beneficially incorporated in other examples. DETAILED DESCRIPTION

[0017] Various features are described below with reference to the accompanying drawings. It should be noted that the drawings may or may not be drawn to scale, and elements of similar structure or function are represented by similar reference numerals in all drawings. It should be noted that the drawings are intended only to facilitate the description of the features. They are not intended to be an exhaustive description of the present specification nor to limit the scope of the claims. In addition, the illustrated examples do not necessarily have all the aspects or advantages shown. The aspects or advantages described in conjunction with a particular example are not necessarily limited to that example, and may be practiced in any other example even if not shown or not so explicitly described.

[0018] Embodiments herein describe a kind of embedded configurable engine, and this embedded configurable engine can solve the low efficiency of CPU discussed above, and does not need the complexity of ISA expansion.In one embodiment, configurable engine is embedded in the cache hierarchy of processor, rather than being embedded in the core of processor.Nevertheless, configurable engine can realize efficient data sharing between main memory, cache memory and core.In addition, engine can be configurable data flow engine, thereby makes it possible to customize for various target domains with same or similar architecture.This is particularly useful for the application with irregular workload.

[0019] In one embodiment, the configurable engine is controlled (or configured) by software adapted to each application domain, such as an operating system (OS). That is, the software can configure the engine according to the data flow profile of the specific application being executed by the processor. In addition, the software can dynamically reconfigure the configurable engine through firmware updates with minimal interference to existing flows.

[0020] Figure 1 An embedded configurable engine 150 in a cache hierarchy of a processor 100 according to an example is illustrated. The processor 100 may be an integrated circuit (or chip) including a plurality of cores 105. Although Figure 1 Four cores 105A-105D are illustrated, but processor 100 may include any number of cores.

[0021] In this example, each core 105 includes an L1 cache 110 and an L2 cache 115. In one embodiment, these caches 110, 115 are private (or non-shared) caches that are not accessible by other cores 105 in the processor 100. Instead, the processor also includes an L3 cache 120 that is shared by the cores 105. That is, any one of the cores 105 can store data and read data from the L3 cache 120.

[0022] In this example, the configurable engine 150 is physically disposed in a portion of the processor 100 that includes the L3 cache 120. In one embodiment, the configurable engine 150 facilitates communication between the L3 cache 120 (and main memory, not shown) and the L2 caches 115A-115D in the cores 105A-105D. For example, the engine 150 may be used with a prefetcher implemented in the engine 150 to retrieve data from the L3 cache 120 (or main memory) and store the data in the L2 cache 115 or L1 cache 110 in the core 105. For example, each core 105 may include a respective prefetcher.

[0023] In the traditional prefetcher model, the prefetcher learns and predicts the data request pattern. However, in the domain-specific model, the prefetcher knows the data layout and use. For example, in a paradigm affected by software, data movement can be extracted from the domain-specific model so that the prefetcher knows the data layout and use without having to learn or predict the pattern. The configurable engine 150 provides a way to use a domain-specific prefetcher to prefetch data from the L3 cache 120 (or main memory) and store the data in the L2 cache 115 for a specific core 105. However, prefetching is only one use of the configurable engine 150. For example, in another embodiment, the configurable engine 150 can be used as a data compression engine. In another example, the configurable engine 150 can observe the data being transferred between the L2 cache and the L3 cache. For example, in order to detect side channel attacks, the engine 150 can observe the state of different data when it is transferred between caches. In another embodiment, the engine observes the data being transferred and affects the program running on the core by modifying the shared data. In another embodiment, the configurable engine 150 may be configured to perform common operations for the system.

[0024] In one embodiment, the configurable engine 150 includes programmable logic circuits (e.g., lookup tables, configurable logic blocks (CLBs), etc.). In one embodiment, the configurable engine 150 may be an FPGA or a coarse-grained reconfigurable architecture (CGRA) embedded in an integrated circuit of the processor 100. In another embodiment, the configurable engine 150 is formed using hardened circuits (such as circuits used to implement a data flow engine). In this case, the configurable engine 150 may not be as flexible as when formed using an FPGA or CGRA, but may still be configurable to perform several different functions (e.g., prefetching or data compression) or enable several different data flow modes. For example, the configurable engine 150 may be a data flow engine, an FPGA fabric, or a bus-configured FPGA fabric.

[0025] Although Figure 1 The configurable engine 150 is illustrated as being embedded in the L3 cache 120, but this is merely an example. In one embodiment, the configurable engine 150 may be embedded in any shared or non-shared cache level. For example, the processor 100 may have an L4 shared cache, in which case the configurable engine 150 may be embedded in that cache. In one embodiment, the configurable engine 150 is embedded in the highest cache memory just below the main memory (e.g., RAM) in the cache hierarchy.

[0026] Figure 2A 2 illustrates a distributed embedded configurable engine 150 in a cache hierarchy of a processor 200 according to an example. Figure 1 The processor 100 in the embodiment of the present invention is an integrated circuit (or chip) including a plurality of cores 105. Figure 2A Four cores 105A-105D are illustrated, but processor 200 may include any number of cores. Figure 2A The core 105 also includes Figure 1 A dedicated L1 cache 110 and a dedicated L2 cache 115 are shown.

[0027] However, in this example, the configurable engine is distributed throughout the L3 cache 120, rather than locating the configurable engine in a continuous portion of the L3 cache 120. As shown, the configurable engine is divided into four configurable engine portions 250A-250D, which are disposed in the L3 cache 120 near the four cores 105A-105D. That is, the configurable engine portion 250A is disposed at an area in the L3 cache 120 near the core 105A, the configurable engine portion 250B is disposed at an area in the L3 cache 120 near the core 105B, the configurable engine portion 250C is disposed at an area in the L3 cache 120 near the core 105C, and the configurable engine portion 250D is disposed at an area in the L3 cache 120 near the core 105D. These individual portions 250 may be viewed as one distributed configurable engine or multiple individual configurable engines.

[0028] In some embodiments, there may be performance benefits to having the circuitry of the configurable engine located in contiguous locations in L3 cache 120, such as Figure 1 However, in other embodiments, there may be performance benefits in distributing circuitry in a configurable engine so that portions 250 are located closer to the corresponding core 105, such as Figure 2A as shown (ie, portion 250 is non-continuous).

[0029] Although Figure 2A The portion 250 of the configurable engine is illustrated as being distributed in the L3 cache 120, but this is merely an example. In one embodiment, the configurable engine 150 may be embedded and distributed in any shared cache level. In one embodiment, the configurable engine portion 250 is embedded and distributed in the highest cache memory just below the main memory (e.g., RAM) in the cache hierarchy.

[0030] In another embodiment, both the distributed configurable engine and the public configurable engine work collaboratively, such as Figure 2BAs shown. For some or all address ranges, each portion of the distributed configurable engines 250A-250D observes memory accesses to and from its associated core 105A-105D. Each portion of the distributed engines 250A-250D may notify the common configurable engine 260 of the observed accesses or forward access requests to the engine 260. The common configurable engine 260 uses this information according to predefined operations for the current workload. The common configurable engine 260 may be interfaced to the cache hierarchy through the cache 270. Although set outside of any core, the cache 270 behaves like the L2 cache of the core, for example, the cache 115A-115D. Such an arrangement allows the engines to be integrated using existing cache coherence mechanisms.

[0031] Figure 3 is a logical view of an embedded configurable engine 150 in a cache hierarchy according to an example. Figure 1 contiguous region of the L3 cache as shown, or if the configuration engines are distributed as shown Figure 2A or Figure 2B Although different non-contiguous regions of the L3 cache are shown, the logical view is the same.

[0032] exist Figure 3 , the core includes an execution unit 305 that sends data to and receives data from an L1 cache 110 and an L2 cache 115. In this example, these caches 110, 115 are private caches that can only be accessed by the execution units in core 105 (and cannot be accessed by other cores in the processor).

[0033] The configurable engine 150 is logically arranged between the L2 cache 115 and the L3 cache 120. That is, the configurable engine 150 manages the data flow between the L2 cache 115 and the L3 cache 120. In addition, Figure 3 The example configurable engine 150 can exchange both control data and application data. Application data may include data used (and generated) by the execution unit 305 when executing a particular user application. Control data exchanged between the engine 150 and the L2 cache 115 allows the engine 150 to query the state of the cache 115 and invalidate specific cache contents. Control data exchanged between the engine 150 and the L3 cache 120 may provide similar functionality. In addition, Figure 3 An L3 cache 120 is illustrated in communication with a main memory 310 (eg, RAM).

[0034] Figure 3Also illustrated is a configurable engine 150 that receives configuration data from the OS 315. The configuration data can configure (or program) the configurable engine to perform a data flow task. In one embodiment, the OS 315 configures the engine 150 to perform a customized data flow task that corresponds to a data flow pattern of a specific user application executed by the execution unit 305. For example, a user application may have an irregular workload that corresponds to an irregular data flow pattern. The OS 315 can configure the engine 150 to perform a data flow task of that pattern, rather than requiring the prefetcher in the core to try to learn the pattern. In this software-centric example, since the OS 315 knows about the irregular data flow pattern, it can configure the engine 150 to perform a data flow task customized for the data flow pattern. Figure 5 Configuring the configurable engine 150 using the OS 315 is discussed in more detail in . Although the embodiments herein discuss configuring the engine 150 using the OS 315, other types of software may be used to configure the engine 150 in a manner that is transparent to the user.

[0035] Figure 3 An example is illustrated in which the engine 150 is located between the dedicated L2 cache 110 and the shared L3 cache 120. The engine 150 can observe the activity of the attached cache in a manner similar to a hardware prefetcher. For example, the engine 150 understands memory accesses and whether the accesses result in misses or hits. The engine 150 can observe and perform many activities like a cache controller. For example, the engine 150 can see and update cache line flags, and determine whether the cache line is filled or evicted. In addition, the engine 150 can perform the following operations on the connection between the L2 cache and the L3 cache: a) intercept, block and / or insert access to the L3 cache, and b) intercept, block and / or insert responses to the L2 cache. These operations allow the configurable engine 150 to manage the movement of data in and out of the core 105 within the cache hierarchy. The engine can operate on the data and put the resulting data back into the cache hierarchy. The engine 150 can be configured to not interfere with some address ranges and not pay attention to memory accesses within those ranges or memory accesses of specific applications. In fact, the engine 150 can interact with the tasks executed on the core 105 at a granular level and can define contexts for these tasks. It is worth noting that this capability exceeds the capabilities of hardware prefetchers or typical L1 / L2 cache controllers.

[0036] Figure 4An embedded configurable engine 150 according to an example is illustrated. The engine 150 includes two components: a common component (e.g., shared circuit 405) and a per-core component (e.g., core-specific circuit 410). The shared circuit 405 is shared between the cores in the processor and other components associated with each core. Within an integrated circuit or die, there may be multiple engines 150. Engines on different dies may collaborate, for example, when an OS migrates a thread from one die to another. The shared circuit 405 may implement multiple engines, and these engines may be assigned to the cores that use them. Among other things, the shared circuit 405 allows tracking activity across multiple cores.

[0037] Per-core components (e.g., core-specific circuitry 410) may provide latency-critical services and implement configurable engines that are specific to the attached core. For example, core-specific circuitry 410A may be specific to one core, while core-specific circuitry 410N may be specific to another core. Even if the configurable engines are Figure 4 405 and core-specific circuits 410 (e.g., similar to Figure 1 ), the functionality may also be physically distributed across multiple parts (or separate engines) across the cache hierarchy (e.g., as shown in Figure 2A and Figure 2B shown).

[0038] In one embodiment, the engine has access to a table lookaside buffer (TLB) 415 to map (or perform address translation) from virtual addresses to physical addresses, and to a reverse TLB to map from physical addresses to virtual addresses. These mappings are advantageous because operations may involve virtual addresses, but memory accesses at the L2 cache or L3 cache use physical addresses. Therefore, the inputs and outputs of the engine 150 may be either physical addresses or virtual addresses.

[0039] Alternatively, two sets of content addressable memories (CAMs) managed directly by the OS can be used for address translation. This has the advantage that the address lookup mechanism of the processor does not need to be changed to take into account the new buffer. In addition, it may be beneficial to allow the engine 150 to access the memory to obtain the intermediate data of the task. In one embodiment, the engine 150 can read the intermediate data from the memory and write the intermediate data to the internal temporary storage memory.

[0040] Figure 5 is a workflow for configuring an embedded configurable engine according to an example. In one embodiment, an instruction scheduler within each core may be responsible for scheduling tasks to run on the engine in the core. However, since the configurable engine discussed above may be embedded in a cache hierarchy, scheduling tasks may be performed differently for the engine.

[0041] In one embodiment, the scheduling of tasks to the configurable engines in the cache hierarchy is completed in two steps. First, the OS (or other types of software) initializes the tasks on the configurable engine based on information from the application. Second, the target activity of the attached cache triggers the task to perform a predetermined operation. In one embodiment, there are two types of activities. One is about the memory access flowing through the engine from the L2 cache or the L3 cache. The access can be of different types, and the engine can be programmed to act on it based on its type (e.g., pre-fetch access, direct write, write back, table walk access) or its target address, that is, different actions are performed according to the range to which the address belongs. Another type of activity is about the activity of the engine 150 observed via the control channel by exchanging control data. For example, if there is a cache miss / hit / fill at L2 / L3 on a specific address range (configured by the OS).

[0042] The high-level interaction between the configurable engine and the OS is described in Figure 5 505. When the OS loads the binary of an application that uses a configurable engine, the OS also configures the configurable engine using the bitstream as shown by arrow 505. In one embodiment, before configuring the engine, the OS first checks whether there is enough space in the configurable engine to implement the task. The application can instruct the OS to wait until the engine becomes available, or to execute without using the engine. The configurable engine can wait for runtime information from the application and wait for the task to become active. That is, the configurable engine can be in an idle state after it is configured, waiting for runtime information and waiting for the task to be activated.

[0043] As shown by arrow 510, the application provides runtime information to the engine 150, which places the engine 150 in a ready state. In one embodiment, the application initiates the flow of runtime information to the engine 150, but the OS is responsible for relaying the runtime information to the engine 150. For example, the application may not be able to interact directly with the engine.

[0044] In one embodiment, the engine may receive multiple information throughout the duration of the application. For example, the application may contain several stages. Therefore, the best prefetch strategy may depend on these stages. For example, early on, a graphics application may process fan-out nodes, but later on process fan-in nodes. These two stages may allow the engine to be configured to monitor different address ranges. In this case, the application will need to at least tell the engine to switch its behavior. In some cases, a task may be active without such information.

[0045] Once a task is activated (e.g., the configuration engine 150 is placed in an active state), the engine 150 monitors and executes the task if predetermined conditions are met. That is, the engine 150 manages data transfer between caches, which may include observing data, moving data, compressing data, processing data, or operating on data.

[0046] The triggering condition may depend on an additional bit on the metadata that the OS sets based on the application's behavior and propagates to each memory access through the page table entry. The bit value is directly associated with the operation or class of operations that will be selected by the engine based on other runtime conditions of the system. These conditions may also include a set of address ranges associated with the operation and transmitted to the engine by other means.

[0047] use Figure 5 In the workflow in the engine 150, the OS can allocate, schedule and load tasks. In addition, the OS can repeat Figure 5 The engine 150 may be configured to perform tasks for a first user application (being executed by the core) during a first time period, but then reconfigured to perform tasks for a second user application during a second time period. The engine may perform several tasks for different applications simultaneously.

[0048] In addition, the OS can perform load balancing on the engines 150 and cores to provide better overall system performance. For example, if there is no room in the engine and the new application has a low priority, the OS can choose to use the engine in the core instead of the configurable engine 150 in the shared cache. In this case, the OS will not configure the engine 150 to perform the task. However, if the new application has a high priority, the OS can use the new implementation of the engine 150 (using Figure 5 ) to replace the current implementation of engine 150 that performs tasks for applications with lower priority, in order to perform tasks for new, higher priority applications.

[0049] Figure 6 It is a flow chart for configuring an embedded configurable engine for use by a user application according to one embodiment. In one embodiment, the software programmer understands the capabilities of the configurable engine and provides an implementation of the operation that will benefit the application as well as a conventional software implementation. The implementation can be obtained from a catalog or built specifically for the application.

[0050] At block 605, a software developer receives or creates a library for an application that will use the configurable engine. An application may contain different stages that use different sets of operations. There may not be enough space in the engine to operate the requested operation. In this case, the OS may fall back to the implementation on the core. Alternatively, if the compiled application uses some functionality from dynamically linked libraries. These libraries may enhance other implementations on the engine. Once the updated library is installed on a system with a configurable engine, the application can be serviced by the engine without recompiling. This is possible due to the nature of dynamic linking.

[0051] At block 610, the application calls an application programming interface (API) to communicate runtime information to the configurable engine. Figure 7 In one embodiment, the library provides an API to further configure the engine to perform different operations when the application is running. Generally speaking, the library allows user applications to interact with the configurable engine using, for example, one or more APIs.

[0052] In one embodiment, the application can use the framework to communicate runtime information to the configurable engine instead of receiving the library. In this case, the application may not have to use the API to communicate with the configurable engine.

[0053] At block 615, the compiler links the library to the application. In one embodiment, multiple applications in a domain (e.g., different user applications) are linked to a shared library. In one embodiment, the library encapsulates a mechanism to communicate with a configurable engine (e.g., which may act as a prefetcher) through the OS kernel using model specific registers (MSRs) or some other method.

[0054] Figure 7 Software code 700 is illustrated for implementing interaction with a configurable engine embedded in a cache hierarchy using a library 705 . Figure 7 An application written to use a domain-specific prefetcher for the graphics analysis domain is illustrated. The application includes a library header, calls to a specific API (e.g., Figure 6 ) and a link to the library ( Figure 6 The library header is in box 615. Figure 7 Marked as "1" in the Figure 7 is marked as "2" in the Figure 7 Marked as "3".

[0055] Figure 8The illustrated scenario is that several applications (e.g., different user applications) of the graphics analysis domain can use a shared library to communicate with an embedded configurable engine. The library encapsulates an OS kernel component that has the privilege to control and monitor the engine through one or more MSRs. Other communication modes can use control registers (CRs), shared addresses, or configuration bits. Libraries from various domains can use common components (e.g., Figure 4 Shared circuit 405 in to interact with the engine.

[0056] The configurable engines discussed above can implement common operations that are beneficial to most applications, such as hardware prefetchers or data compression. In such cases, the configuration of the engine can be performed by the firmware at system startup, even before the OS becomes online. Updated versions of the operations can be delivered through firmware updates. The firmware of systems with the same configurable engine can be different. The system operator can update the firmware that is suitable for the current use case. For example, a system that is mainly used as a search engine can use one firmware version of the configurable engine, while another system that is used as a database system can use another firmware version of its configurable engine.

[0057] The embodiments of this paper allow to accelerate tasks that are tightly coupled with those tasks in the core, because these tasks share the same data path, that is, the cache hierarchy. On the other hand, the embodiments of this paper may not destroy the already challenging core design. Different devices use the same core design to include different variations of configurable engines for different use cases without changing the core itself. When the engine is set in the core itself, the engine is started and controlled by the special instructions of the ISA. Therefore, the benefits of this type of engine are limited to operations compatible with ISA restrictions, that is, the maximum number of operands. On the contrary, applications can interact with the configurable engines embedded in the cache hierarchy through OS and libraries. Therefore, any application can benefit by providing suitable operations, without affecting other applications like changing ISA. As a result, the embodiments of this paper describe easy-to-implement technology to provide a specific distinguishing feature of a vendor. A domain-specific framework can be used to further simplify user interaction. This type of framework encapsulates the interaction with the engine in a way that the user does not have to directly adopt an API.

[0058] In addition, the embodiments of the present invention may allow tasks to be run closer to memory to reduce latency and / or energy. The engine can observe and operate on data without propagating it all the way to the core (or before propagating it all the way to the core). As an example, this latency advantage enables the engine to implement a prefetcher for the graphics analysis domain that improves the performance of the application by up to 6%. In addition, almost half of the core pipeline cycles are stopped due to data acquisition and instruction acquisition. Using the embodiments of the present invention, the amount of instructions and data acquired into the core can be reduced, which means that compared to existing solutions, the embodiments of the present invention not only allow larger operations to be implemented efficiently, but also help improve overall system performance.

[0059] Some systems place configurable engines away from the core through NoC-like interconnects on or off-chip. Therefore, the latency between the engine and the core is high. In addition, frequent communication between the engine and the core uses memory bandwidth and reduces overall system performance. As a result, such integration can only provide acceleration to tasks that have little communication with the core. A known use case is for tasks in which the core configures the operation engine and the input location and output location. The input or output of the engine may be directly connected to the input / output (IO) circuit. The core will be notified when the operation ends. These embodiments of the present invention propose methods and system structures that allow tasks that are more closely coupled to the core to be accelerated. In particular, the structure allows this to be done without negatively affecting the core design. This makes it possible to take advantage of the locality provided by the cache hierarchy and still operate near the main memory as required by the domain. This impact will solve the inefficiency of data movement outside the existing load / store paradigm.

[0060] Some other solutions include a programmable prefetcher unit (PPU) for accelerating applications with irregular workloads. However, the PPU cannot control and set the context of the tasks running on the core. The PPU can only help to introduce data earlier. On the contrary, using the embodiments of this article, the OS can store read-only data in the engine. When the core reads the data, the engine can provide the data and prevent access propagation, thereby saving memory bandwidth. However, the PPU will need to introduce data. In addition, the PPU is located inside the core itself, rather than in a shared cache memory. In addition, the PPU is not on the instruction acquisition path and therefore cannot operate on the instruction stream. In addition, the PPU uses a traditional processor architecture, and the embodiments of this article propose a data flow engine that can be more energy efficient for read-modify-write operations of typical workloads.

[0061] Some other solutions use engines for the memory services programming model, where the purpose of the system is to bring computation closer to the data. As a result, the engines are small and distributed, making implementation less practical due to the additional configurability overhead. In contrast, the embodiments of this article can use larger engines to address the inefficiency of data movement to the core by configuring the engine to target the domain running on the core. In addition, the method of using a configurable engine is different from the engine used for the memory services programming model.

[0062] Some other solutions use general engines to provide optimization of data movement. These general engines use callbacks to allow software to modify cache behavior, where callbacks are short-term tasks created and scheduled entirely by the hardware in the engine. On the contrary, the engine herein can use OS to allocate, schedule and load tasks. Therefore, in addition to the core, OS can also perform load balancing on the engine to provide better overall system performance. For example, if there is no space in the engine and the priority of the new application is low, OS can choose to implement the task that is not running on the configurable engine in the shared cache on the core. However, if the priority is high, OS can replace the implementation of the engine with a lower priority running application with the implementation of its corresponding core to make room for the new application. In addition, the general engine can rely on phantom addresses (virtual addresses that are never mapped to physical addresses and only exist in cache) to trigger the engine. On the contrary, in one embodiment, the configurable engine discussed herein can use hardware-only solutions to not rely on phantom addresses, and avoid OS changes to support them.

[0063] In the foregoing, reference is made to the embodiments presented in the present disclosure. However, the scope of the present disclosure is not limited to the specifically described embodiments. Instead, any combination of the described features and elements (whether or not involving different embodiments) is contemplated as implementing and practicing the contemplated embodiments. In addition, although the embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether a particular advantage is achieved by a given embodiment does not limit the scope of the present disclosure. Therefore, the foregoing aspects, features, embodiments and advantages are merely illustrative and are not considered to be elements or limitations of the appended claims unless expressly stated in the claims.

[0064] As will be appreciated by those skilled in the art, the embodiments disclosed herein may be embodied as systems, methods, or computer program products. Thus, various aspects may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or a combination of software and hardware implementations, all of which may be generally referred to herein as "circuits," "modules," or "systems." Additionally, various aspects may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0065] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the foregoing. More specific examples (not a complete list) of computer-readable storage media would include the following: an electrical connector having one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium is any tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device or apparatus.

[0066] A computer readable signal medium may include a propagated data signal having computer readable program code embodied therein (e.g., in baseband or as part of a carrier wave). Such a propagated signal may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0067] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0068] The computer program code for performing operations for various aspects of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as "C" programming language or similar programming languages). The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider).

[0069] Aspects of the present disclosure are described below with reference to flowchart illustrations and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments presented in the present disclosure. It should be understood that each frame of the flowchart illustration and / or block diagram and the combination of frames in the flowchart illustration and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that instructions executed by a processor of a computer or other programmable data processing device create a mode for implementing the function / action specified in one or more frames of a flowchart and / or block diagram.

[0070] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other device to function in a specific manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0071] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operating steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer-implemented method, so that the instructions executed on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0072] The flow chart and block diagram in the figure show the possible concrete implementation architecture, functionality and operation of the system, method and computer program product according to various examples of the present invention.In this regard, each frame in the flow chart or block diagram may represent the module, segment or part of one or more executable instructions for implementing the specified logical function of the instruction.In some alternative implementations, the function pointed out in the frame may not occur in the order pointed out in the figure.For example, depending on the function involved, the two frames shown continuously can actually be performed substantially at the same time, or these frames can sometimes be performed in reverse order.It will also be noted that the combination of each frame in the block diagram and / or the flow chart illustration and the frame in the block diagram and / or the flow chart illustration can be implemented by a system based on special hardware that performs a specified function or action or performs a combination of special hardware and computer instructions.

[0073] While the foregoing is directed to particular examples, other and further examples may be devised without departing from the basic scope of the invention, and the scope of the invention is determined by the following claims.

Claims

1. A processor, comprising: a first core comprising a first dedicated cache in a first level of a cache hierarchy; a second core comprising a second dedicated cache in the first level of the cache hierarchy; a shared cache located in a second level of the cache hierarchy, the shared cache comprising a configurable engine including circuitry configured to manage data passed between the first dedicated cache and the second dedicated cache and the shared cache, The configurable engine is capable of being reconfigured by software to perform tasks for different user applications executing on the first core and the second core.

2. The processor of claim 1 , wherein the software is an operating system (OS), and reconfiguring the configurable engine using the OS comprises: When the OS loads a binary of one of the different user applications, the configurable engine is configured using a bitstream provided by the OS.

3. The processor of claim 2, wherein after configuring the configurable engine, reconfiguring the configurable engine using the OS comprises: maintaining the configurable engine in an idle state until runtime information is received from the one of the different user applications; providing the runtime information to the configurable engine to place the configurable engine in a ready state; as well as The configurable engine is activated so that when a predetermined condition is met, the configurable engine executes the task.

4. The processor of claim 1 , wherein the configurable engine is configured to perform interception, blocking, and insertion of accesses to at least one of (i) the first and second dedicated caches or (ii) the shared cache.

5. The processor of claim 1, wherein the configurable engine is disposed in a contiguous portion of the shared cache.

6. The processor of claim 1, wherein the configurable engine comprises a plurality of portions, the plurality of portions being distributed in non-contiguous areas in the shared cache, wherein a first portion of the portions is disposed proximate to the first core, and a second portion of the portions is disposed proximate to the second core.

7. The processor of claim 6, wherein the configurable engines comprise a common configurable engine, wherein the plurality of portions of the configurable engines notify the common configurable engine of accesses observed at the respective cores or forward access requests to the common configurable engine.

8. The processor of claim 1, wherein the configurable engine is configured to access a memory to convert addresses between physical memory addresses and virtual memory addresses when sending data between the first and second dedicated caches and the shared cache.

9. The processor of claim 1, wherein the configurable engine is configured to read intermediate data from a memory and write the intermediate data to an internal scratch memory, and write a final result to the memory.

10. The processor of claim 1, wherein the software is an operating system, wherein the operating system is configured to perform load balancing for the configurable engines based on priorities of the different user applications.

11. The processor of claim 1, wherein the configurable engine is one of: a data flow engine, an FPGA fabric, or a bus-configured FPGA fabric.

12. A method comprising: providing a configurable engine in a shared cache of a processor, wherein the processor includes a plurality of cores having dedicated caches, wherein the configurable engine includes circuitry to manage data passed between the dedicated caches and the shared cache; receiving a library or framework for the configurable engine; providing runtime information from a user application to the configurable engine using the library or framework; as well as The library or framework is linked to the user application.

13. The method of claim 12, wherein the library permits the user application to interact with the configurable engine, the method further comprising: An application programming interface (API) is called to provide the runtime information to the configurable engine, wherein the API is provided by the library.

14. The method according to claim 12, further comprising: The library is linked to a plurality of applications, wherein the library permits the plurality of applications to communicate with the configurable engine through an OS kernel.

15. The method according to claim 12, further comprising: The configurable engine is used to prefetch data or perform data compression on behalf of the user application.

16. The method of claim 12, wherein the configurable engine is disposed in a contiguous portion of the shared cache.

17. The method of claim 12, wherein the configurable engine comprises a plurality of portions, the plurality of portions being distributed in non-contiguous areas in the shared cache, wherein a first portion of the portions is disposed proximate to a first core in the processor, and a second portion of the portions is disposed proximate to a second core in the processor.

18. The method of claim 17, wherein the configurable engines comprise a common configurable engine, wherein the plurality of portions of the configurable engines notify the common configurable engine of accesses observed at respective cores or forward access requests to the common configurable engine.

19. The method according to claim 12, further comprising: When data is sent between the private cache and the shared cache, memory is accessed by the configurable engine to convert addresses between physical memory addresses and virtual memory addresses.

20. The method according to claim 12, further comprising: Load balancing of the configurable engines is performed based on priorities of different user applications.