Quality of service support for input / output and other agents

Abstract

Techniques for quality of service (QoS) support for input / output devices and other agents are described. In an embodiment, a processing device includes execution circuitry to execute a plurality of software threads; hardware for controlling monitoring or allocation of one or more shared resources among the plurality of software threads; and configuring the storage to enable monitoring or allocation of the one or more shared resources between the plurality of software threads and between the one or more channels through which the one or more devices are used to connect to the one or more shared resources.

Description

Background Art

[0001] On computers and other information processing systems, various techniques can be used to provide various levels of quality of service (QoS) to clients, applications, and the like. For example, processor cores in a multi-core processor may use shared system resources such as caches (e.g., last level cache or LLC), system memory, input / output (I / O) devices, and interconnects. The QoS provided to applications may be degraded and / or unpredictable due to contention for these or other shared resources. Some processors include processors such as those from Technologies such as the company's Resource Director Technology (RDT) that enable visibility and / or control over how shared resources such as LLC and memory bandwidth are being used. For example, such technologies may be useful for controlling applications that may over-utilize memory bandwidth relative to their priority. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Various examples according to the present disclosure will be described with reference to the accompanying drawings, in which:

[0003] Figure 1 A processor supporting input / output (I / O) proxy quality of service (QoS) capabilities is illustrated according to an embodiment.

[0004] Figure 2A A system including an I / O memory management unit (IOMMU) according to an I / O QoS-based marking method is illustrated.

[0005] Figure 2B The diagram includes an embodiment of a central processing unit (CPU) agent and a non-CPU agent connected via a fabric.

[0006] Figure 2C Illustrated is a non-CPU proxy QoS feature enable register according to an embodiment.

[0007] Figure 2D Devices marked with a resource monitoring identifier (RMID) and / or a class of service (CLOS) according to an embodiment are shown.

[0008] Figure 2E An I / O architecture according to an embodiment is shown.

[0009] Figure 2F A system is shown in which an I / O block includes a mapping table that maps RMID and CLOS values ​​to channels mapped to devices for I / O traffic on an interconnect, according to an embodiment.

[0010] Figure 2G A mask for controlling I / O QoS according to an embodiment is shown.

[0011] Figure 2H , Fig.2I , Figure 2J , Figure 2K , Figure 2L , Figure 2M and Figure 2N An architectural model for I / O QoS control according to an embodiment is shown.

[0012] Fig.2O , Figure 2P and Figure 2Q A programming interface for I / O QoS according to an embodiment is shown.

[0013] Figure 3A , Figure 3B , Figure 3C and Figure 3D An example of Advanced Configuration and Power Interface (ACPI) mapping for I / O QoS according to an embodiment is shown.

[0014] Figure 3E , Figure 3F , Figure 3G , Figure 3H and Fig. 3I An example of a table for ACPI-based I / O QoS according to an embodiment is shown.

[0015] Figure 4 An example computing system is illustrated.

[0016] Figure 5 A block diagram of an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller is shown.

[0017] Fig. 6A is a block diagram illustrating both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to an example.

[0018] Figure 6Bis a block diagram illustrating both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example.

[0019] Figure 7 An example of execution unit(s) circuitry is shown.

[0020] Figure 8 is a block diagram illustrating converting binary instructions in a source instruction set architecture to binary instructions in a target instruction set architecture using a software instruction converter according to an example. DETAILED DESCRIPTION

[0021] The present disclosure relates to methods, apparatus, systems and non-transitory computer-readable storage media for input / output (I / O) agents and other agents to convert quality of service (QoS) support. According to some examples, an apparatus includes: an I / O agent and a processor core, the processor core being used to provide quality of service features for use by the I / O agent.

[0022] As mentioned in the background section, the processor may include, for example, Technologies such as the company's Resource Deployment Technology (RDT) that enable visibility and / or control of how shared resources such as LLC and memory bandwidth are being used. Aspects, implementations and / or techniques related to such technologies involving monitoring, measuring, estimating, tracking, etc. of memory bandwidth usage may be referred to as "memory bandwidth monitoring" or "MBM" (which may also be used to refer to a memory bandwidth monitor, hardware / firmware / software for performing memory bandwidth monitoring, etc.), however, the embodiments are not limited to the use of this term. Aspects, implementations and / or techniques related to such technologies involving allocating, limiting, throttling, providing availability, etc. of memory bandwidth may be referred to as "memory bandwidth allocation" or "MBA" (which may also be used to refer to the amount of memory bandwidth allocated, made available, to be allocated, etc.), however, the embodiments are not limited to the use of this term.

[0023] Additionally or alternatively, a processor or execution core in an information handling system may support a cache allocation technique that includes a cache capacity bit mask. For example, The RDT feature set provides an allocation (resource control) capability set that includes cache allocation technology (CAT) supported by various cache levels, including level 2 (L2) and level 3 (L3) caches. CAT enables the OS, hypervisor, VMM, or similar system service management agent to specify the amount of cache space that an application can fill by programming the cache capacity bitmask (CBM).

[0024] Embodiments may include techniques implemented in hardware (e.g., in circuitry, in silicon, etc.) involving QoS support for I / O agents and other agents. Use of embodiments may be expected to provide visibility and / or control of utilization of shared resources by I / O devices (such as Peripheral Component Interconnect Express (PCIe) and Compute Express Link (CXL) devices), with new capabilities for enabling monitoring / control of usage by any agent in a system using such shared resources.

[0025] For convenience, the embodiments described below may refer to the agent as an I / O agent, but any such reference to an I / O agent may mean any agent (e.g., an I / O device, an integrated accelerator, a CXL device, a field-programmable gate array (FPGA), a storage device, an agent other than a central processing unit (non-CPU agent), etc.). For convenience, any or all of these technologies may be referred to as I / O QoS, IO QoS, non-CPU agent QoS, I / O RDT, IO RDT, non-CPU agent RDT, etc., but the embodiments are not limited to I / O devices or RDTs.

[0026] Figure 1 A processor supporting input / output (I / O) proxy quality of service (QoS) capabilities as further described below is illustrated in accordance with an embodiment. According to an embodiment, I / O QoS may be implemented in a processor, processor core, execution core, etc., which may be any type of processor / core, including a general-purpose microprocessor / core, such as Processor series or from A processor / core from another processor family of Intel® or another company, a special purpose processor or microcontroller, or any other device or component in an information handling system in which embodiments may be implemented.

[0027] For example, Figure 1 A processor or processor core 100 (which may represent any of the following) supporting I / O QoS capabilities according to an embodiment is illustrated. Figure 4 The processor 470, 480 or 415 in the system 400, Figure 5 processor 500 or one of cores 502A to 502N in the embodiment of the present invention, and / or Figure 6B For convenience and / or example, some features (e.g., instructions, registers, etc.) may be implemented by reference to a particular processor architecture (e.g., and / or IA32), but the embodiments are not limited to these features, names, architectures, etc.

[0028] As shown, processor 100 includes an instruction unit 110, a configuration storage device (eg, model or machine specific registers (MSRs)) 120, execution unit(s) 130, and any other elements not shown.

[0029] The instruction unit 110 may be used in conjunction with Figure 6B The front end unit 630 corresponds to and / or is implemented / included in Figure 6B The front end unit 630 in, and / or may include any circuit systems, logic gates, structures and / or other hardware (such as an instruction decoder) to obtain, receive, decode, interpret, schedule and / or handle instructions or programming mechanisms, such as a processor identification instruction to be executed and / or processed by the processor 100 (e.g., as described below and represented as CPUID of box 112) or other means (e.g., via an advanced configuration and power interface or ACPI), and one or more read or write instructions (e.g., as described below and represented as RDMSR / WRMSR of box 114) or other means (e.g., via a memory mapped I / O (MMIO) interface). Figure 1 In , instructions that may be decoded or otherwise handled by instruction unit 110 are represented as blocks with dashed boundaries because the instructions themselves are not hardware, but instruction unit 110 may include hardware or logic capable of decoding or otherwise handling these instructions.

[0030] Any instruction format may be used in embodiments; for example, an instruction may include an opcode and one or more operands, where the opcode may be decoded into one or more microinstructions or micro-operations for execution by an execution unit. The operands or other parameters may be associated with the instruction implicitly, directly, indirectly, or in any other manner.

[0031] Configuration storage 120 may include any one or more MSRs or other registers or storage locations, one or more of which may be in the core, one or more of which may be within the core or outside the core (e.g., in an uncore, a system agent, etc.) to control processor features, control and report processor performance, handle system-related functions, etc. In various embodiments, one or more of these registers or storage locations may or may not be accessible to applications and / or user-level software, may be written to or programmed by software, a basic input / output system (BIOS), etc.

[0032] In an embodiment, the instruction set of processor 100 may include instructions for accessing (e.g., reading and / or writing) MSRs or other storage devices, such as instructions for reading from or writing to MSRs (RDMSR, WRMSR), and / or instructions for reading from or writing to or programming other registers or storage locations, including via MMIO.

[0033] In an embodiment, configuration storage 120 may include one or more MSRs, fields or portions of MSRs, or other programmable storage locations, such as described below and / or in the Figure 2C , Figure 2G , Fig.2O , Figure 2P , Figure 2Q , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Fig. 3I According to an embodiment, the processor 100 may also include a mechanism for indicating support and enumeration of capabilities. For example, in response to an instruction (e.g., In the processor, CPUID instruction), one or more processor registers (e.g., EAX, EBX, ECX, EDX) can return information to indicate whether the capabilities according to the embodiments are supported, to what extent the capabilities according to the embodiments are supported, how the capabilities according to the embodiments are supported, etc.

[0034] The execution unit(s) 130 may be used in conjunction with the following Figure 6B The execution engine 650 or Figure 7 The execution unit circuit system 662 in the embodiment corresponds to and / or is implemented / included in Figure 6B The execution engine 650 or Figure 7 In the execution unit circuit system 662.

[0035] Figure 2A A system 200A including an I / O memory management unit (IOMMU) 210A according to an IOMMU-based marking method for I / O QoS is illustrated. In the system 200A, the IOMMU 210A includes a mapping table 212A for mapping resource monitoring identifiers (RMIDs) and class of service (CLOS) values ​​to devices such as device 222A, PCIe solid state drive (SSD) 224A, and network interface controller (NIC) 226A for I / O traffic on an interconnect 214A through a connection 220A.

[0036] In contrast, embodiments may provide for dynamically setting QoS priorities that associate each resource with a specific QoS tag for monitoring (RMID) and controlling (CLOS) shared resources, and mapping device PCIe / CXL traffic channels (TCs) to virtual channels (VCs), and further to RMID / CLOS pairs. In an I / O complex, implementations may include a mapping table from device traffic to these tags. Implementations may include architectural elements such as ACPI tables for enumeration (referred to as "IRDT"), and MMIO interfaces, as described below.

[0037] Embodiments may include tag-based per-device or per-TC / VC or per-RMID or per-device class monitoring or control of shared resources such as cache space. Embodiments may include per-device / tag / class monitoring of shared resource usage such as cache occupancy used by a device, or "overflow" memory bandwidth or direct memory access (DMA) memory bandwidth used by a device.

[0038] In this detailed description, a processor or execution core (e.g., Architecture A thread running on a CPU-based architecture (e.g., an IA core) may be referred to as a CPU agent, and embodiments may provide QoS functionality for non-CPU agents, the term non-CPU agents broadly covering a collection of agents, excluding CPU agents (e.g., an IA core) that read from and write to a cache or memory, such as PCIe / CXL devices and integrated accelerators.

[0039] Figure 2B The illustration includes an embodiment of a CPU agent 202B and a non-CPU agent PCIe I / O block 204B, a CXL interface 206B, and other agents 208B connected via a high-speed fabric 200B.

[0040] Embodiments may include and / or provide capabilities for monitoring and controlling resource utilization of non-CPU agents, including PCIe and CXL devices, and integrated accelerators. In embodiments, non-CPU agent features enable monitoring of I / O device shared cache and memory bandwidth and cache allocation control by marking device channels (PCIe / CXL TC / VC) for monitoring / allocation, respectively, using tags applied in I / O blocks, without involving IOMMUs or process address space identifiers (PASIDs). Embodiments may enable I / O devices to have the same resource utilization as CPU agents. Capabilities Cache Monitoring Technology (CMT), Memory Bandwidth Monitoring (MBM) and Cache Allocation Technology (CAT) equivalent capabilities.

[0041] In an embodiment, CMT provides visibility into the cache (typically L3 or LLC). CMT provides occupancy counters for non-CPU agents on a per-RMID basis, so cache occupancy (e.g., capacity used by a particular RMID for I / O agents) can be dynamically tracked and read back during system operation.

[0042] In an embodiment, L3 aggregation and local external MBM allow system software to monitor the usage of bandwidth between L3 cache and local or remote memory on a per-RMID basis by non-CPU agents.

[0043] In an embodiment, CAT allows for non-CPU agents to control shared cache capacity on a per-CLOS basis, thereby achieving both isolation and overlap for better throughput, fairness, determinism, and differentiation.

[0044] In some cases, embodiments may include or provide control at a device level and / or channel level granularity. This granularity may be coarser than that for software threads. A CPU core may execute hundreds of threads, all of which may be tagged with RMID and CLOS, while an I / O device such as a NIC may service hundreds of software threads, but it may only be monitored and controlled at the device level or channel level (see subsequent sections for details on channel level monitoring and control).

[0045] Example enumeration for I / O RDT monitoring (e.g., non-CPU agent RDT feature enumeration)

[0046] The CPU-agent RDT feature uses the CPUID instruction to enumerate supported features and supported levels, and uses the architecture model specific registers (MSRs) as an interface to monitor and allocate features.

[0047] In an embodiment, the non-CPU proxy RDT is built on the CPU proxy by extending the CPUID to indicate the presence and integration of the non-CPU proxy RDT, and providing rich enumeration information in a vendor-specific extension to ACPI (e.g., in an I / O RDT (IRDT) table). An embodiment provides a mechanism for understanding the following: the structure of devices attached to a particular link behind an I / O block, and what forms of tagging are supported on a per-link basis. For example, the rich enumeration information cited above may include information about the following: supported features, the structure of devices attached to a particular link behind an I / O block, supported tags and control forms on each link, and a specific MMIO interface for controlling a given device.

[0048] In an embodiment, software may use the existing CPUID leaf to collect the maximum number of RMID and CLOS tags for each resource level (e.g., L3 cache), and non-CPU proxy QoS may also be subject to these limitations. Some platforms may support a mix of features, for example, supporting L3 CAT and non-CPU proxy QoS equivalents, but not supporting CMT or MBM monitoring. In an embodiment, software may parse both CPUID and ACPI before attempting to use non-CPU proxy QoS to gain a detailed understanding of platform support and capabilities.

[0049] In an embodiment, I / O QoS may use one or a combination of CPUID-based enumeration and ACPI-based enumeration (IRDT table). In an embodiment, when support for non-CPU agent RDT features is detected using CPUID, ACPI may be consulted for further details on the level of feature support, the device structure behind the various I / O ports, and the specific MMIO interface used to control a given device.

[0050] CPUID-based enumeration can provide a method by which all architectural RDT features can be enumerated. For the CPU-proxy RDT, monitoring details can be enumerated in the CPUID subleaf denoted as CPUID.0xF.[ResID], where ResID corresponds to the resource ID bit index from the CPUID.0xF.0 subleaf. Similarly, RDT allocation features are described in CPUID.0x10.[ResID]. Note that the ResID bit positions are not guaranteed to be symmetric or have the same encoding.

[0051] In an embodiment, a bit may be added in the CPU agent RDT CMT / MBM leaf: CPUID.0xF.[ResID=1]:EAX[bits 9, 10]; EAX[bit 9] being set may indicate the presence of non-CPU agent cache occupancy monitoring (equivalent to the CMT feature of the CPU agent RDT); EAX[bit 10] being set may indicate the presence of non-CPU agent memory L3 external BW monitoring (equivalent to the MBM feature of the CPU agent RDT); a new bit may be provided in the L3 CAT leaf CPUID.0x10.[ResID=1(L3 CAT)]:ECX[bit 1]; ECX[bit 1] may be set to indicate the presence of non-CPU agent cache allocation technology (equivalent to the L3 CAT feature of the CPU agent RDT); as described above, ECX[bit 2] may define that L3 code / data prioritization (CDP) is supported if it is set. Note that if the device does not have the capability to fill into the core L2 cache, the equivalent bit is defined in CPUID.0x10.[ResID=2(L2CAT)].

[0052] If any of these non-CPU agent RDT enumeration bits are set, indicating the presence of the monitor feature or the allocation feature, it also indicates the presence of the IA32_L3_IO_RDT_CFG architecture MSR. As described below, this MSR can be used to enable the non-CPU agent RDT feature.

[0053] Some platforms may support a mix of features, such as supporting L3 CAT architectural control and non-CPU agent RDT equivalents, but not supporting CMT / MBM monitoring or non-CPU agent monitoring equivalents, and these capabilities should be enumerated on a per-function and per-platform basis.

[0054] In an embodiment, there may be no CPUID leaf or sub-leaf created for non-CPU agent QoS; instead, the existing CPUID leaf may be augmented or extended, for example, where a bit per resource type indicates whether there is non-CPU agent RDT monitoring or control. For example, CPUID.0xF (shared resource monitoring enumeration leaf).[ResID=1]:EAX[bits 9, 10] enumerates for the presence of CMT features and MBM features of non-CPU agents, respectively; CPUID.0x10 (cache allocation technology enumeration leaf).[ResID=1 (L3 CAT)]:ECX[bit 1] enumerates for the presence of L3 CAT features of non-CPU agents.

[0055] In an embodiment, if a specific CPU-surrogate RDT feature is not present, an attempt to use a non-CPU-surrogate RDT equivalent may result in a general protection fault in the MSR interface. An attempt to enable an unsupported feature in an I / O complex may cause writes to the corresponding MMIO enable or configuration interface to be ignored.

[0056] Non-CPU Agent RDT Feature Enable MSR

[0057] In an embodiment, before configuring the non-CPU proxy RDT through MMIO, the feature should be enabled using the non-CPU proxy RDT feature enable MSR IA32_L3_IO_RDT_CFG (e.g., MSR address 0C83H), an example of which is shown in Figure 2C The presence of one or more CPUID bits indicating support for one or more non-CPU agent RDT features also indicates the presence of this MSR. This MSR can be used to enable the non-CPU agent RDT feature.

[0058] In an embodiment, two bits are defined in MSR 200C. For example, if the CPUID indicates that one or more non-CPU agent RDT resource allocation features are present, then the L3 non-CPU agent RDT allocation enable bit (e.g., bit 0, shown as IRAE or A 202C) is supported and, when set, enables the non-CPU agent RDT resource allocation function. For example, if the CPUID indicates that one or more non-CPU agent RDT resource monitoring features are present, then the L3 non-CPU agent RDT monitoring enable bit (e.g., bit 1, shown as IRME or M 202C) is supported and, when set, enables the non-CPU agent RDT monitoring feature.

[0059] In an embodiment, the default value of MSR 200C is 0x0, which specifies that both features are disabled by default. All undefined bits are reserved. Writing a non-zero value to any reserved bit will generate a general protection fault (#GP(0)).

[0060] In an embodiment, MSR 200C is scoped at the L3 cache level and is cleared at system reset. It is expected that software will configure MSR 200C consistently across all L3 caches that may be present on the package.

[0061] In the example of a device marked with RMID and / or CLOS, such as Figure 2D As shown in , PCIe devices 210D and CXL devices 212D are marked for monitoring and controlling upstream resources in L3 or LLC 202D (within fabric 200D). For CPU cores 220D, and as defined in the CPU agent RDT feature set, their bandwidth can be controlled with the MBA feature set. Memory controller 222D (coupled to double data rate and / or high bandwidth memory 224D) and ultra path interconnect (UPI) 226D can also be connected to LLC 202D through fabric 200D. In an embodiment, the cores, PCIe devices, and CXL devices can be arranged symmetrically around the fabric and can be symmetrical in their ability to use RMID and CLOS.

[0062] In an embodiment, the RDT monitoring data retrieval MSRs IA32_QM_EVTSEL and IA32_QM_CTR are used to monitor usage by non-CPU agents in the same manner as they are used for the RDT of a CPU agent.

[0063] In an embodiment, the CPU cache capacity control MSR interface is also used to control I / O device access to the L3 cache. The CLOS assigned to the device and the corresponding capacity bit mask in the IA32_L3_QOS_MASK_n MSR govern the portion of the L3 cache into which data may be filled.

[0064] In an embodiment, the CLOS tag retains the same meaning with respect to L3 fill for both CPU agents and non-CPU agents. Other cache levels may also be applicable, depending on model-specific data flow patterns that are governed by how I / O device data is filled into the cache in a model-specific manner, such as by the implementation of the Data Direct I / O (DDIO) feature of a given product generation.

[0065] Public Tags

[0066] In an embodiment, the non-CPU agent RDT allows the traffic and operations of the non-CPU agent to be associated with RMID and CLOS. In the CPU agent RDT, RMID and CLOS are numerical tags that can be associated with the operation of a thread through the IA32_PQR_ASSOC MSR. In the non-CPU agent RDT, a series of MMIO interfaces can be defined and used to enable devices and / or channels to be tagged with RMID and / or CLOS, and can be defined and used to associate I / O devices with RMID and CLOS tags, and the numerical interpretation of the tags remains the same.

[0067] For example, from the perspective of a CPU core or a non-CPU agent, a particular CLOS tag (such as CLOS [5]) may mean the same thing, and the same is true for RMID. In this way, RMID and CLOS for non-CPU agents are considered to be drawn from a common pool of RMID or CLOS tags, defined at a common L3 configuration level. Typically, these tags have specific meanings at a specific level of resources (such as L3 cache).

[0068] For the non-CPU agent RDT, specific devices can be selected for monitoring and control, and software enumeration and control are added to enable the non-CPU agent RDT to build on top of the CPU agent RDT to understand the device topology behind the I / O link (such as PCIe or CXL) and enable association of devices with RMID tags and CLOS tags.

[0069] I / O blocks and channels

[0070] In an embodiment, an I / O interface block is used to bridge from an in-order, non-coherent domain (such as PCIe) to an out-of-order, coherent domain (e.g., a shared interconnect fabric hosting an L3 cache). The non-CPU agent RDT interface describes the devices connected behind each I / O complex (which may contain a downstream PCIe root port or CXL link) and enables configuration RMID / CLOS tags for them.

[0071] Examples of I / O architectures are Figure 2E Channel mapping can occur between devices and I / O blocks or anywhere within an I / O block.

[0072] For example, in Figure 2E200E, a PCIe device 208E may be connected via a root port 206E and routed via an I / O block 204E, which applies non-CPU proxy RDT tags (RMID and CLOS tags) before the traffic reaches a coherent fabric 202E. Device traffic routed on various TCs and mapped to VCs may be mapped to internal channels between the root ports and the I / O blocks as defined in the PCIe specification. The non-CPU proxy RDT enumeration structure defines the mapping between PCIe VCs and non-CPU proxy RDT channels, allowing software to perform tag configuration based on the channels for platforms that support this capability (see the following section for more details).

[0073] For example, in Figure 2E As shown in architecture 210E, CXL.IO and / or CXL.Cache links 216E may connect CXL device 218E to I / O block 214E responsible for tagging (if supported) before traffic reaches coherence fabric 212E. The links (CXL.IO and CXL.Cache) are controlled separately through separate software interfaces.

[0074] Note that this implementation is different from the previous approach because the I / O block tags a limited number of channels with RMID / CLOS - the IOMMU based implementation is no longer used to associate PASID with RMID / CLOS.

[0075] I / O Block Configuration

[0076] As described in the previous section, a PCIe device mapped to a channel through its VC can be configured on a per-channel basis in an I / O block. CXL is a subset example of this, with the same configuration format but with only one configuration entry (the equivalent of a single channel).

[0077] In an embodiment, an enumerated number of channels are supported in the IRDT ACPI and configured through the MMIO interface. Several downstream PCIe or CXL devices can be mapped to various channels and their traffic flows can be marked (if applicable) through the configuration of the I / O block.

[0078] For example, Figure 2F The illustrated system 200F includes an I / O block 210F including a mapping table 212F for mapping resource monitoring identifier (RMID) and class of service (CLOS) values ​​for I / O traffic on an interconnect 214F to channels mapped to devices, such as device 222F, PCIe SSD 224F, and NIC 226A, connected to the I / O block 210F via connection 220F.

[0079] Shared L3 configuration

[0080] The following subsections describe embodiments including the interaction between shared L3 configuration and non-CPU agent RDT features.

[0081] Software Flow

[0082] In an embodiment, the software actions required to utilize the non-CPU proxy RDT include enumeration of supported capabilities and details of that support, as well as use of these features through the architecture platform interface. Software can enumerate the presence of the non-CPU proxy RDT through a combination of parsed bit fields from the CPUID and IRDT ACPI tables. The CPUID infrastructure provides basic information about the level of support for the existing CPU proxy RDT and the non-CPU proxy RDT, as well as details of the common CLOS / RMID tags shared with the CPU proxy RDT. The IRDT ACPI extension specifically provides more details about the non-CPU proxy RDT, such as which I / O blocks support the non-CPU proxy RDT, and where the control interface for configuring the I / O blocks is located in the MMIO space.

[0083] In an embodiment, after software has enumerated the presence of a non-CPU agent RDT, configuration changes may be made by selecting a subset of RMID / CLOS tags for use with the non-CPU agent RDT, and resource limits on those tags through MSRs for shared platform resources such as L3 cache (e.g., for I / O usage of L3 CAT) may be configured through the I / O block MMIO interface (whose location is enumerated via the IRDT ACPI). After resource limits are associated, RMID / CLOS tags may be applied to I / O device upstream traffic by assigning each I / O device to an RMID / CLOS tag through its mapping to channels (and through corresponding configuration of the MMIO interface for each I / O block).

[0084] In an embodiment, while upstream shared SoC resources like L3 cache are monitored and controlled via shared RMID / CLOS tags, certain resources closer to the I / O can be controlled locally within each I / O block. In this approach, RMID and CLOS are used for upstream resources that can be shared with the CPU core, but capabilities unique to the I / O device domain are controlled through I / O block specific interfaces.

[0085] In an embodiment, after tags are assigned and resource limits are applied, upstream traffic from I / O devices is monitored and controlled within the shared resources of the SoC through I / O blocks tagged with corresponding RMID / CLOS, just as CPU-agent resources are controlled against these tags in the CPU-agent RDT.

[0086] In an embodiment, since the IRDT ACPI table used to enumerate the non-CPU agent RDT is generated by the BIOS, in case of hot-plug operation, the OS or VMM software should update its internal tracking of device mappings based on the newly added or removed devices.

[0087] In some embodiments including bifurcation of a set of PCIe lanes, downstream devices that may be mapped to individual lanes may still be individually tagged and controlled, but devices sharing a lane will be mapped together against the same RMID / CLOS tag. Since CXL devices have no concept of lanes, all downstream devices will be constrained by the same RMID / CLOS in the case of a bifurcated CXL link.

[0088] Monitoring: Data flow for RMID

[0089] As previously described, after RMID tags are applied to non-CPU agent traffic, all RMID-driven counter infrastructure in the platform can be used with the non-CPU agent RDT. For example, RMID-based cache occupancy and memory bandwidth overflow data is collected for non-CPU agents and can be retrieved by software. For each supported cache monitoring resource type, the hardware supports only a limited number of RMIDs. CPUID.(EAX=0FH (shared resource monitoring enumeration leaf), ECX=1H).ECX enumerates the highest RMID value that can be monitored using this resource type.

[0090] In an embodiment, since an interface for CPU-proxy RDT data retrieval for RMID-based counters has already been defined, the same interface is used, including MSR-based data retrieval, for corresponding sets of three event IDs (EvtIDs) defined for the CMT and MBM features of the CPU-proxy RDT.

[0091] In an embodiment, RMIDs are assigned to devices by software from a pool of RMIDs defined at the L3 cache level, and the IA32_QM_EVTSEL / IA32_QM_CTR MSRs may be used to specify the RMID and event ID and retrieve the data.

[0092] In embodiments where attributes are inherited from the CPU-agent RDT, the appropriate MSR pairs may be used to retrieve event data.All access rules and usage order, reserved bit attributes, initial values, and virtualization attributes may be inherited from the CPU-agent RDT.

[0093] Distribution: CLOS-based control interface

[0094] In an embodiment, the allocation feature for non-CPU agents uses CLOS-based tags for control of cache at a given level, which is subject to the location of data filling from I / O devices in a specific cache and SoC implementation, which in the common case may be the last level cache (L3) as described in ACPI (e.g., specifically, in the IRDT subtable and its flags called resource control structure (RCS)). Software can adjust the level of cache it controls based on the expected level(s) of cache into which I / O data can be filled, subject to flags in the corresponding RCS. This in turn can affect which CPU agent CAT control mask software program controls data filling for non-CPU agents, and may vary depending on how the specific RCS is connected to shared resources on the platform.

[0095] In an embodiment, for each supported cache allocation resource type, the hardware supports only a limited number of CLOS. CPUID.(EAX=10H (cache allocation technology enumeration leaf), ECX=2):EDX[15:0] reports the maximum CLOS supported for the resource (CLOS is zero referenced, which means that a reported value of "15" would indicate a total of 16 supported CLOS). Bits 31:16 are reserved.

[0096] For example, for a non-CPU agent (such as a PCIe device) to fill data into the L3 cache, the "cache level bit vector" of the RCS would have bit 17 set to indicate the L3 cache, and software could control the CPU agent RDT L3 CAT mask (in IA32_L3_QoS_MASK_nMSR) to define the amount of cache that the non-CPU agent can fill into. As with RMID management, the CLOS used in this context is drawn from a pool of applicable resources (in this context, the L3 cache).

[0097] If other cache levels are introduced or used in the future, incremental software enablement may be required to understand the fills into the other cache levels.

[0098] In an embodiment, the mask for control may be extracted from an existing definition of such cache control in a CPU-agent RDT definition (e.g., details such as reserved fields, initialization values, etc.), such as a CPU-agent RDT L3 CAT control MSR 200G, which may be programmed by privileged software 210G, such as Figure 2G as shown in .

[0099] CXL-Specific Considerations

[0100] The following subsections describe CXL-specific device considerations according to embodiments, including management of traffic on multiple links and CXL device types.

[0101] CXL Block Interface Basics

[0102] In an embodiment, a CXL device may be connected to a resource management unit descriptor (RMUD) (eg, an I / O RDT RMUD) via multiple RCSs and may involve independent control of each RCS.

[0103] In an embodiment, the non-CPU agent RDT feature provides monitoring and control for CXL.IO and CXL.cache link types; however, CXL.mem is not subject to control in the I / O block because it is considered a resource rather than an agent. As previously described, and where supported, bandwidth to CXL.mem can be controlled at the agent source (e.g., using an MBA).

[0104] In an embodiment, an accelerator (eg, an integrated accelerator using an integrated CXL link) may be monitored and controlled using the semantics described in the previous sections.

[0105] Use Cases

[0106] Examples of non-CPU proxy RDT use cases involving PCIe, CXL, and integrated accelerators are described below. In these examples and other embodiments, RMID and CLOS tags can be configured and activated by software.

[0107] As an implementation of the above architecture model, Figure 2H As shown in , I / O block 220H tags upstream DMA traffic (such as PCIe writes) so that utilization of the fabric's shared resources 230H (such as L3 cache) by device 200H can be monitored and controlled via RMID and CLOS, which are mapped to channels by channel mapping block 210H.

[0108] Fig.2IAn example is shown with a high-performance PCIe SSD 200I that is subject to tagging by an I / O block 220I using CLOS (e.g., so that its L3 cache footprint can be controlled) and RMID (e.g., so that its L3 cache occupancy and overflow bandwidth to memory can be monitored), which are mapped to channels by a channel mapping block 210I for monitoring and control of shared resources 230I (e.g., L3 cache).

[0109] Figure 2J An example of a CXL device 200J is shown where two paths are used for traffic to a device sharing a resource 230J, one path through CXL.IO and one path through CXL.Cache, through two separate I / O blocks 220J and 222J, respectively, each corresponding to a channel mapping block 210J or 212J. Note that the CXL.Cache link defines only one channel. In such a case, software can configure the RMID flag and CLOS flag for the link individually. The links operate independently. Also note that no control is provided for CXL.Mem, because CXL.Mem accesses memory on the target device, and the bandwidth from the logical processor can be controlled using the memory bandwidth allocation (MBA) feature of the RDT. A more detailed discussion of this case is provided below.

[0110] Figure 2K An example of multiple devices with different attributes accessing a shared resource 230K is shown. A pair of PCIe devices 200K and 202K (each corresponding to a channel mapping block 210K or 212K) on separate I / O blocks 220K and 222K, respectively, can be independently controlled with separate RMID tags and CLOS tags. In this case, a PCIe SSD 200K that does not effectively utilize cache may be restricted, but a NIC 202K that fills into cache to obtain data consumed by a CPU core may be prioritized (e.g., set to 25% compared to 5% for the SSD).

[0111] Figure 2L An example of accessing shared resources 230L is shown where a CXL accelerator 200L (e.g., a CXL-enabled field programmable gate array (FPGA) card) utilizes CXL.IO and CXL.Cache controlled by I / O blocks 220L and 222L (corresponding to channel mapping blocks 210L and 212L) independently of an I / O block 224L (corresponding to channel mapping block 214L) to which a PCIe device 202L is attached.

[0112] Figure 2MAn example of tagging and controlling an integrated accelerator 200M (e.g., a data streaming accelerator (DSA)) that accesses a shared resource 230M through an I / O block 220M and a channel mapping block 210M and a PCIe device 202M that accesses a shared resource 230M through an I / O block 222M and a channel mapping block 212M is shown. Depending on system load conditions and DSA usage, software may choose to allocate non-overlapping portions of cache to minimize cache contention effects.

[0113] Figure 2N A complex example with multiple features in use is shown. Access to shared resources 230N by various PCIe devices 202N is controlled by I / O blocks 224N and channel mapping blocks 214N using non-CPU proxy RDT, but there are also CXL devices 200N using CXL.IO and CXL.Mem. CXL devices 200N can be tagged and controlled by I / O blocks 220N and 222N and channel mapping blocks 210N and 212N on their CXL.IO interfaces.

[0114] The primary purpose of CXL.Mem is for host access to device memory, however, traffic responses through the CXL.Mem path are not subject to MBA bandwidth shaping, although these traffic responses are sent with RMID tags and CLOS tags. If bandwidth is constrained on this link and software seeks to reallocate bandwidth across different priorities of access agents (such as CPU cores), the MBA feature can be used to reallocate bandwidth and throttle at the source of the request (traffic injection point of the agent).

[0115] This example shows that for comprehensive management of cache and bandwidth resources on a platform, a combination of CPU-agent RDT and non-CPU-agent RDT control may be necessary.

[0116] In an embodiment, the programming interface for I / O counter width, overflow bit, CMT, MBM, etc. enumerations for I / O RDT monitoring can be shared with existing features, such as using Fig.2O The register 200O shown in FIG. 200A can be shared with existing features: CPUID.(EAX=0FH, ECX=1H).EAX[bit 9]: If 1, indicates the presence of non-CPU agent RDT CMT support. CPUID.(EAX=0FH, ECX=1H).EAX[bit 10]: If 1, indicates the presence of non-CPU agent RDT MBM support. CPUID.(EAX=0FH, ECX=1H).EAX[7:0]: In bits [7:0], the counter width is encoded as an offset from 24b. In EAX bits 7:0, the counter width is encoded as an offset from 24b. A value of zero in this field implies support for 24-bit counters. A value of 8 indicates support for 32-bit counters, as in the third generation First introduced in the Xeon Scalable processor family, although other implementations may vary. With this enumerable counter width, the requirement for software polling at 1Hz is removed. Software can poll at different rates with reduced risk of rollover, and under typical conditions, rollover may require hundreds of seconds (although this value is not explicitly specified and may change and decrease in future processor generations as memory bandwidth increases). If software seeks to ensure that rollover does not occur more than once between samples, sampling at 1Hz while consuming data for the enumerated counter width can provide this feature for a particular platform and counter width. CPUID.(EAX=0FH, ECX=1H).EAX[8]: Enumeration of the presence of the overflow bit in the IA32_QM_CTR MSR via EAX bit [8]. Software using the MBM Event Retrieval MSR interface should be updated to understand this new format, which enables the top 61-bit MBM counters to be provided by future platforms, with Error, Unavailable, and Overflow bits to indicate error conditions. It is expected that higher level software that consumes the resulting bandwidth value will not be affected. If CPUID.0F.[ECX=1]:EAX[bit 8] is set, the overflow bit is defined in bit 61 of the IA32_QM_CTR MSR. If supported, this rollover bit will be set on overflow of the MBM counter and will be reset on read, enabling variable software defined counter polling intervals for reduced sampling overhead. Bits 31:11 of EAX are reserved.

[0117] In an embodiment, after the monitoring and sub-signatures have been enumerated, the software may use Figure 2P The register 200P shown in 200P associates a given software thread (or multiple threads as part of an application, virtual machine (VM), application group, or other abstraction) with an RMID to begin using the monitoring feature. Similar concepts can be used to mark I / O device channels, as described elsewhere in this specification.

[0118] In an embodiment, the CMT / MBM data retrieval interface (e.g., Figure 2Q200Q) can be shared with existing RDT functionality.

[0119] Embodiments may provide a programming interface for I / O RDT allocation. For example: CPUID.(EAX=10H, ECX=1): ECX[bit 1]: If 1, indicates that L3CAT for non-CPU agents is supported. CPUID.(EAX=10H, ECX=1): ECX[bit 3]: If 1, indicates that the non-contiguous capacity bit mask for L3 CAT is supported, which means that the bits set by software in the various IA32_L3_MASK_n registers are not required to be contiguous. This capability is supported simultaneously with I / O RDT L3 CAT. Bit 0 and bits 31:4 of ECX are reserved.

[0120] ACPI-based enumeration and definition for I / O RDT

[0121] In an embodiment, software can query the processor support for shared resource monitoring and allocation capabilities by executing CPUID for the CPU agent RDT feature. Further details of the enhanced feature support for non-CPU agents can be consulted to the ACPI structure named IRDT. These ACPI structures also provide the location of the specific MMIO interface for allocating or monitoring shared resources.

[0122] In an embodiment, the IRDT ACPI enumeration definitions and RMID / CLOS tags and mappings (e.g., to SoC components) may provide: Top-level configuration information for the SoC, such as how many RMID / CLOS tags are supported by the non-CPU-agent RDT relative to the CPU-agent RDT (enumerated by CPUID) · Logical description of the control hierarchy - meaning which MMIO address is used to configure the link's RMID / CLOS flags Flexibility in implementation topology of devices behind the I / O blocks, covering cases with discrete or integrated PCIe and CXL links and integrated accelerators Enhanced usability information for software (including device topology, TC / VC / channel mapping information for advanced QoS usage) to enable forward compatibility

[0123] In an embodiment, the top-level ACPI enumeration structure defined to support I / O RDT is the IRDT structure, which is a vendor-specific extension to the ACPI table space. The named IRDT structure is generated by the BIOS and contains all other non-CPU agent RDT ACPI enumeration structures and fields (e.g., as described below), and may define new ACPI layouts, mappings to RMUDs, device specific structures (DSS), RCS, etc. In an embodiment, reserved fields in the IRDT structure should be initialized to 0 by the BIOS.

[0124] Embodiments may include an RMUD under or embedded within an IRDT structure.RMUDs are typically mapped to I / O blocks within the system, although it is possible for one RMUD to be defined at other levels, such as one RMUD per SoC.

[0125] Figure 3A An example mapping is shown in FIG. 3 , which shows ACPI details 300A at the top and a SoC (e.g., SoC) mapping 300B. The depicted relationship between the IRDT table 302A and the RMUD 304A is for a typical implementation, where the RMUD 304A describes the properties of the I / O blocks (e.g., 312A, 314A, 316A). The IRDT table 302A defines zero or more RMUDs 304A, and the RMUD 304A contains one or more RPs.

[0126] In an embodiment, RMUD structure 304A contains two embedded structures, DSS 306A and RCS, which map to devices and links and help describe the relationship about which I / O devices are connected to specific links, and which I / O links are used by which devices. Each RMUD 304A defines one or more DSS 306A and RCS.

[0127] exist Figure 3A In the example of , there is one DSS 306A for each PCIe device (e.g., 320A controlled by I / O block 312A, or any PCIe endpoint (EP) controlled by I / O block 316A), CXL device (e.g., 330A controlled by I / O block 314A), or other non-CPU agent device (e.g., accelerator) subordinate to the RMUD. CXL devices may be expected to have multiple links (e.g., CXL.Cache and CXL.IO), and this topology is described by the associated DSS and multiple RCSs for the device and its links. Note that Figure 3AThe DSS 306A is shown downstream of the RMUD 304A, but for simplicity, the RCS is not shown.

[0128] Figure 3B An example of a mapping of RMUD 302B in ACPI top level 300B to DSS 304B and one or more RCS 306B is shown. Each device 320B attached to an I / O block 312B in SoC level 310B is described by DSS 304B and has one or more links whose properties are described in RCS 306B. RCS 306B contains pointers to MMIO locations (e.g., in absolute address form, rather than relative to base address registers (BARs)), allowing software to configure RMID / CLOS tags and bandwidth shaping properties (if supported) in I / O blocks 312B.

[0129] Figure 3C An example with further layers of detail is shown; a device 320C mapped by an I / O block 312C in the SOC level 310C is described by an RMUD 302C in the ACPI top level 300C, a DSS 304C describes the properties of the device 320C, and an RCS 306C provides a pointer to an MMIO location 308C that is used to configure marking and bandwidth shaping for a specific link.

[0130] Given the above table hierarchy, Figure 3D An example CXL Type 1 (CXL.IO+CXL.Cache) device mapping is shown in FIG. At the ACPI top level 300D, a device 320D is described by one DSS 306D behind an RMUD 304D, while two RCSs 308D and 309D are used, one for each link type CXL.IO and CXL.Cache corresponding to I / O blocks 312D and 314D at the SoC level 310D.

[0131] In view of the previously described ACPI table hierarchy and the relationship between RMUD, DSS, RCS, etc., examples of the formats and constituent field definitions of IRDT table 300E, RMUD table 300F, DSS table 300G, RCS table 300H, and MMIO table 31 are shown in FIG. Figure 3E , Figure 3F , Figure 3G , Figure 3H and Fig. 3I The explanations, corner cases, interactions between fields, etc. are described below.

[0132] Figure 3EAn example of a top-level ACPI table structure (I / O Resource Boot Technology Table (IRDT) 300E) is shown in FIG, and an instance of the table is defined at the system level and generated by the system BIOS. The table includes a unique signature and the length of all substructures (including embedded RMUD). The length of the IRDT table is variable.

[0133] A series of high-level flags allow basic capabilities for monitoring and control of I / O links (e.g., PCIe) and coherent links (e.g., CXL) to be quickly extracted. Embedded within the IRDT tables is a set of one or more RMUDs, which are typically mapped to I / O blocks and define their properties. In some instantiations, one RMUD may be defined for the system, or in a more fine-grained approach, one RMUD may be defined for each downstream link and device combination, although this is expected to be an uncommon case.

[0134] Figure 3F An example of an RMUD table structure 300F is shown in FIG. RMUD structure 300F includes a number of fields, including: the length of the RMUD instance and all embedded substructures (DSS and RCS entries); integration parameters mapped to SoC attributes, including minimum and maximum RMID and CLOS tags that can be used when monitoring and controlling devices under this RMUD. While it is common that these parameters will match the CPU agent RDT parameters, there may be some RMUDs that support a subset of the entire RMID and CLOS space.

[0135] Each RMUD entry contains several embedded DSS and RCS identified by their "Type" fields which describe the device and link behind the given RMUD.

[0136] The device scope structure behind each RMUD describes the properties of the device, that is, each DSS maps 1:1 with the device behind a specific RMUD.

[0137] Figure 3G An example of a DSS 300G is shown in . The DSS table definition includes a type field (type = 0 identifies a DSS), the length of the entry, the device type, and an embedded channel management structure (CHMS). The CHMS defines which RCSs are applicable to controlling this device (DSS), and which internal I / O block channels each of the link's virtual channels (VCs) can be mapped to (in the case of PCIe, up to eight VCs are supported, but in the case of CXL, only the first entry is valid). A valid configuration for the CHMS includes one entry for each RCS (link).

[0138] In the DSS device type field, a value of 0x02 indicates that the PCIe sub-hierarchy is described by this DSS. Each root port described by the DSS will have type 0x02. System software can use the enumerated devices found under such root ports to understand the shared bandwidth relationship in the channels under RMUD.

[0139] DSS type 0x01 indicates the presence of a root complex integrated endpoint device (RCEIP) such as an accelerator. Note that the PCI sub-hierarchy may represent a root port, and for each DSS corresponding to a root port, device type = 0x02 is expected.

[0140] Note that the CHMS field contains a list of CHMS structures, which may, for example, describe DSS entries capable of sending traffic over multiple channels (which in turn are described by unique RCS entries).

[0141] Note that no discrete pluggable devices (e.g., PCIe cards) are directly described by DSS entries, instead, the root port is indicated (device type 0x02).

[0142] exist Figure 3H An example of an RCS 300H is shown in FIG. 10. The RCS provides details of the monitoring and control types supported for a particular link interface type (such as PCIe or CXL), and the MMIO locations where tables exist that can be used to apply monitoring and control features. The MMIO locations provided are absolute locations in the MMIO space (64 bits), rather than being hosted in a specific device and defined relative to a BAR.

[0143] Note that if CXL.IO and PCIe devices share the bandwidth of a certain RCS and its lanes, traffic for both protocols is carried on the same lane entry.

[0144] Note that in the enumeration, the fields, RMID offset and CLOS offset are specified relative to the "RCS block MMIO location" field, which means that the RMID offset and CLOS offset may be relocatable within the MMIO space. The offset defines a block of contiguous sets of RMID or CLOS tag fields, and the number of entries is defined by the "lane count" field (e.g., a value of 8 lanes may be common in some PCIe tag implementations).

[0145] MMIO register description for I / O RDT

[0146] In an embodiment, the non-CPU agent RDT related register set (MMIO interface) can reside on at least one 4KB aligned memory mapped page. The exact location of the register area is implementation dependent and is transmitted to the system software by the BIOS through the IRDT ACPI structure. Multiple RCSs can be mapped to the same 4KB aligned page or different pages. No other unrelated registers can exist in the page for the non-CPU agent RDT. A virtual machine monitor (VMM) or operating system can use page-based access control to ensure that only specified entities can use the non-CPU agent RDT control.

[0147] In an embodiment, when accessing a non-CPU proxy RDT MMIO interface, note that writes to reserved fields, writes to reserved offsets within the MMIO space, or writes to values ​​greater than the supported maximum value for a field will be ignored by the hardware.

[0148] In an embodiment, software interacts with the non-CPU agent RDT feature by reading memory mapped registers and writing memory mapped registers. Software access to these registers includes: When registers are updated via multiple accesses (either in software or due to hardware teardown), the accesses should be ordered to ensure proper behavior. These should be documented as part of the respective register description. Lock operations on non-CPU agent RDT related registers may not be supported. Software should not issue lock operations on non-CPU agent RDT feature hardware registers.

[0149] Link interface type RMID / CLOS tag MMIO interface

[0150] In an embodiment, the IRDT ACPI structure may define an MMIO interface for configuring RMID / CLOS for each link interface type, as defined in the RCS. The MMIO pointer defined in the RCS field describes the location where the configuration interface exists for a specific link interface type. The MMIO location is defined in absolute address terms.

[0151] Fig. 3I An example MMIO field layout for RMID and CLOS marking and bandwidth shaping in MMIO table 300I is shown. A common format is used for all RCS types, including, for example, RCS instances supporting PCIe or CXL use the same field layout.

[0152] In some embodiments, the RDT RMID / CLOS tag can be placed in the MMIO for software independent configuration. In other embodiments, an intermediate tag type can be defined, which is later mapped to a RMID / CLOS pair (or similar monitoring / allocation pair). In other embodiments, the monitoring / allocation tags can be combined.

[0153] An embodiment may include a common table format for enumerating MMIOs across all RCSs. In an embodiment, the MMIO table format, fields, etc. may be as described below.

[0154] For example, as shown in Table 300I, RMID / CLOS may be defined as separate MMIO blocks, in other embodiments they may be 1:1 interleaved.

[0155] Note that the RCS::REGW field indicates the register access width of the field, either 2 bytes or 8 bytes.

[0156] Note that the cardinality of the RMID and CLOS fields is enumerated in the RCS, and the size of these fields varies with the number of channels supported. The set of configurable RMIDs and CLOS is organized as a contiguous block of 4B (4-byte) registers.

[0157] The "PQR" field starting at the enumeration offset (RCS::CLOS Block Offset) is defined with the enumerated register field spacing of RCS::REGW, which may require a 2B or 8B register access. There is a CLOS register block followed by a RMID register block, indexed on a channel-by-channel basis. That is, setting the value in the IO_PQR_CLOS0 field will specify that CLOS is used for channel [0] on this RCS.

[0158] The effective field widths for RMID and CLOS are defined via the CPUID leaf for shared-L3 configurations.

[0159] If supported, the higher offset allows multiple lanes to be programmed (higher than channel 0). Given that PCIe supports multiple VCs, multiple lanes may be supported in the case of a PCIe link, but a CXL link only supports two entries, one in the table for IA_PQR_CLOS0 and one in the table for IO_PQR_RMID0.

[0160] The RMID field and the CLOS field are interpreted as numeric tags, just as they are in the CPU-agent RDT feature set, and software can assign RMID and CLOS as needed.

[0161] Software can reconfigure the RMID field value and the CLOS field value at any point during runtime, and these values ​​can be read back at any time. Since all architecture CPU agent RDT infrastructure is also dynamically reconfigurable, this enables control loops to work collaboratively and consistently across capability sets.

[0162] The following describes software architectural considerations, programming guidelines, recommended usage flows, and related considerations for RDT features for non-CPU agents according to embodiments, which can build on the architectural concepts and software usage examples discussed above.

[0163] In an embodiment, software seeking to use RDT for non-CPU agents may have several tasks to understand. For example: • Enumeration of capabilities for CPU-agented RDTs (via CPUID) and enumeration of capabilities for non-CPU-agented RDTs (via CPUID and ACPI). • Reservation of RMID and CLOS (or understanding of the sharing implications of using RMID and CLOS) from pools available at each resource level and subject to best practices for RMID and CLOS management on the specific processor. • Pre-configuration of any resource limits used to regulate device activity, such as cache masks for CLOS intended for the device. Configuration of tag attributes for each device through the MMIO interface described by the ACPI structure, such as associating the device with a specific RMID, CLOS, and bandwidth limit (if applicable). Enable the RDT feature for non-CPU agents by enabling the MSR infrastructure (IA32_L3_IO_QoS_CFG MSR) at MSR address 0xC83. • Periodically adjust resource limits subject to software policy and any control loops that may exist. Understand the implications of Sub-NUMA clustering (SNC), if present and enabled.

[0164] Example Apparatus, Methods, etc.

[0165] According to some examples, an apparatus (eg, a processing system) includes an input / output agent and a processor core to provide quality of service features for use by the input / output agent.

[0166] According to some examples, a processing device (e.g., a processor core and execution core, a processor, a system, a SoC, etc.) includes: an execution circuit system for executing multiple software threads; hardware for controlling monitoring or allocation of one or more shared resources between the multiple software threads; and a configuration storage device for enabling monitoring or allocation of one or more shared resources between the multiple software threads and between one or more channels, through which one or more devices are used to connect to the one or more shared resources.

[0167] Any such examples may include any of the following aspects or any combination thereof. The configuration storage device is also used to associate a quality of service tag with multiple software threads for monitoring or allocating one or more shared resources between the multiple software threads, and wherein one or more channels are also used to associate with the quality of service tag for monitoring or allocating one or more shared resources between the one or more channels. The quality of service tag includes a resource monitoring identifier. The quality of service tag includes a service category value. One or more shared resources include a shared cache. One or more shared resources include bandwidth to a memory. One or more devices include one or more input / output devices. One or more devices include one or more accelerators. One or more devices include a peripheral component interconnect express device. One or more devices include a computing express link device. One or more channels are used to be mapped to one or more devices using one or more advanced configuration and power interface data structures. The configuration storage device is also used to associate a quality of service tag with one or more channels.

[0168] According to some examples, a method includes: enabling, by programming configuration storage in a processing device, monitoring or allocation of one or more shared resources between multiple software threads and between one or more channels, through which one or more devices are used to connect to the one or more shared resources; and controlling the monitoring or allocation of the one or more shared resources between the multiple software threads and between the one or more channels during execution of the multiple software threads by the processing device.

[0169] Any such examples may include any of the following aspects or any combination thereof. The method includes: by programming a configuration storage device in a processing device, associating a quality of service tag with multiple software threads for monitoring or allocating one or more shared resources between the multiple software threads, wherein one or more channels are also used to be associated with the quality of service tag for monitoring or allocating one or more shared resources between the one or more channels. The quality of service tag includes a resource monitoring identifier or a service category value. One or more shared resources include a shared cache or bandwidth to a memory. One or more devices include one or more input / output devices, one or more accelerators, one or more peripheral component interconnect express devices, or one or more compute express link devices. The method includes: mapping one or more channels to one or more devices by configuring one or more advanced configuration and power interface data structures.

[0170] According to some examples, a system includes: one or more input / output devices; and a processing device, the processing device including: an execution circuit system, the execution circuit system is used to execute multiple software threads; hardware, the hardware is used to control the monitoring or allocation of one or more shared resources between the multiple software threads; and a configuration storage device, the configuration storage device is used to enable the monitoring or allocation of one or more shared resources between the multiple software threads and between one or more channels, through which the one or more input / output devices are used to connect to the one or more shared resources.

[0171] Any such examples may include any of the following aspects or any combination thereof. One or more input / output devices include one or more peripheral component interconnect express devices or one or more computing express link devices. The configuration storage device is also used to associate a quality of service tag with multiple software threads for monitoring or allocation of one or more shared resources between multiple software threads, and wherein one or more channels are also used to associate with the quality of service tag for monitoring or allocation of one or more shared resources between one or more channels. The quality of service tag includes a resource monitoring identifier. The quality of service tag includes a service category value. One or more shared resources include a shared cache. One or more shared resources include bandwidth to a memory. One or more channels are used to be mapped to one or more devices using one or more advanced configuration and power interface data structures. The configuration storage device is also used to associate a quality of service tag with one or more channels.

[0172] Any such examples may include any of the aspects described above or below and / or illustrated in the accompanying drawings, or any combination thereof.

[0173] According to some examples, a device may include means for performing any function disclosed herein; a device may include a data storage device storing code that, when executed by a hardware processor or controller, causes the hardware processor or controller to perform any method or any part of a method disclosed herein; the device, method, system, etc. may be as described in the detailed description; a method may include any method that can be performed by a device according to an embodiment; a non-transitory machine-readable medium may store instructions that, when executed by a machine, causes the machine to perform any method or any part of a method disclosed herein. Embodiments may include any details, features, etc. or combinations of details, features, etc. described in this specification.

[0174] Example Computer Architecture

[0175] The following is a detailed description of an example computer architecture. Other system designs and configurations for laptops, desktop computers, handheld personal computers (personal computers, PCs), personal digital assistants, engineering workstations, servers, decomposed servers, network devices, network hubs, switches, routers, embedded processors, digital signal processors (digital signal processors, DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices known in the art are also appropriate. In a word, various systems or electronic devices that can include processors disclosed herein and / or other execution logics are generally appropriate.

[0176] Figure 4 An example computing system is illustrated. Multiprocessor system 400 is an interfaced system and includes multiple processors or cores including a first processor 470 and a second processor 480 coupled via an interface 450, such as a point-to-point (PP) interconnect, fabric, and / or bus. In some examples, first processor 470 and second processor 480 are isomorphic. In some examples, first processor 470 and second processor 480 are heterogeneous. Although example system 400 is shown as having two processors, the system can have three or more processors, or can be a single processor system. In some examples, the computing system is a system on a chip (SoC). .

[0177] Processors 470 and 480 are shown as including integrated memory controller (IMC) circuitry 472 and 482, respectively. Processor 470 also includes interface circuits 476 and 478; similarly, second processor 480 includes interface circuits 486 and 488. Processors 470, 480 may exchange information via interface 450 using interface circuits 478, 488. IMCs 472 and 482 couple processors 470, 480 to respective memories, namely memory 432 and memory 434, which may be portions of main memory locally attached to the respective processors.

[0178] Processors 470, 480 may each use interface circuits 476, 494, 486, 498 to exchange information with a network interface (NW I / F) 490 via respective interfaces 452, 454. Network interface 490 (e.g., one or more of an interconnect, bus, and / or fabric, and in some examples, a chipset) may optionally exchange information with a coprocessor 438 via interface circuit 492. In some examples, coprocessor 438 is a special-purpose processor such as, for example, a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processing unit (GPGPU), a neural-network processing unit (NPU), an embedded processor, and the like.

[0179] A shared cache (not shown) may be included in either processor 470, 480, or external to both processors but connected to the processors via an interface such as a PP interconnect, such that if the processors are placed in a low power mode, local cache information of either or both processors may also be stored in the shared cache.

[0180] The network interface 490 can be coupled to the first interface 416 via the interface circuit 496. In some examples, the first interface 416 can be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, the first interface 416 is coupled to a power control unit (PCU) 417, which can include circuit systems, software, and / or firmware to perform power management operations with respect to the processors 470, 480, and / or the coprocessor 438. The PCU 417 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 417 also provides control information to control the generated operating voltage. In various examples, the PCU 417 can include various power management logic units (circuitry systems) to perform hardware-based power management. Such power management may be entirely processor controlled (e.g., controlled by various processor hardware and may be triggered by workload and / or power constraints, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (such as a platform or power management source or system software).

[0181] PCU 417 is illustrated as existing as logic separate from processor 470 and / or processor 480. In other cases, PCU 417 may execute on a given one or more cores of a core (not shown) of processor 470 or 480. In some cases, PCU 417 may be implemented as a microcontroller (dedicated or general purpose) or other control logic that is configured to execute its own dedicated power management code (sometimes referred to as P-code). In yet other examples, the power management operations to be performed by PCU 417 may be implemented outside the processor, such as by a separate power management integrated circuit (PMIC) or another component external to the processor. In yet other examples, the power management operations to be performed by PCU 417 may be implemented within BIOS or other system software.

[0182] Various I / O devices 414 and a bus bridge 418 can be coupled to the first interface 416, and the bus bridge 418 couples the first interface 416 to the second interface 420. In some examples, one or more additional processors 415 are coupled to the first interface 416, such as a coprocessor, a high-throughput integrated many-core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor. In some examples, the second interface 420 can be a low pin count (LPC) interface. Various devices can be coupled to the second interface 420, including, for example, a keyboard and / or mouse 422, a communication device 427, and a storage circuit system 428. The storage circuit system 428 can be one or more non-transitory machine-readable storage media as described below, such as a disk drive or other mass storage device, which in some embodiments may include instructions / codes and data 430. Further, an audio I / O 424 can be coupled to the second interface 420. Note that other architectures are possible besides the point-to-point architecture described above. For example, a system such as multiprocessor system 400 may implement a multi-drop interface or other such architecture instead of a point-to-point architecture.

[0183] Example core architectures, processors, and computer architectures

[0184] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of these cores may include: 1) general-purpose in-order cores, for general-purpose computing purposes; 2) high-performance general-purpose out-of-order cores, for general-purpose computing purposes; 3) special-purpose cores, mainly for graphics and / or scientific (throughput) computing purposes. Implementations of different processors may include: 1) a CPU, including one or more general-purpose in-order cores for general-purpose computing purposes and / or one or more general-purpose out-of-order cores for general-purpose computing purposes; and 2) a coprocessor, including one or more special-purpose cores mainly for graphics and / or scientific (throughput) computing. These different processors result in different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor on a separate die in the same package as the CPU; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as a dedicated core); and 4) a system on a chip (SoC), which may be included on the same die with the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0185] Figure 5 A block diagram of an example processor and / or SoC 500 is illustrated, which may have one or more cores and an integrated memory controller. The processor 500 illustrated in solid line blocks has a single core 502(A), a system agent unit circuitry 510, and a set of one or more interface controller unit circuitry 516, while the optionally added dashed line blocks illustrate an alternative processor 500 as having multiple cores 502(A)-(N), a set of one or more integrated memory control unit circuitry 514 in the system agent unit circuitry 510, and dedicated logic 508, and a set of one or more interface controller unit circuitry 516. Note that the processor 500 may be Figure 4 One of the processors 470 or 480 or the coprocessors 438 or 415.

[0186] Thus, different implementations of processor 500 may include: 1) a CPU, where the dedicated logic 508 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 502(A)-(N) are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, or a combination of the two); 2) a coprocessor, where cores 502(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 502(A)-(N) are a large number of general purpose in-order cores. Thus, processor 500 may be a general purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 500 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as, for example, complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0187] The memory hierarchy includes one or more levels of cache unit circuitry 504(A)-(N) within the cores 502(A)-(N), a set of one or more shared cache unit circuitry 506, and external memory (not shown) coupled to the set of integrated memory controller unit circuitry 514. The set of one or more shared cache unit circuitry 506 may include one or more intermediate level caches, such as level 2 (L2), level 3 (L3), level 4 (4), or other levels of cache, such as last level cache (LLC), and / or combinations thereof. While in some examples an interface network circuitry 512 (e.g., a ring interconnect) provides an interface to dedicated logic 508 (e.g., integrated graphics logic), the set of shared cache unit circuitry 506, and the system agent unit circuitry 510, alternative examples use any number of well-known techniques to provide interfaces to such units. In some examples, coherency is maintained between one or more circuitry in the shared cache unit circuitry 506 and the cores 502(A)-(N). In some examples, interface controller unit circuitry 516 couples core 502 to one or more other devices 518, such as one or more I / O devices, storage, one or more communication devices (e.g., wireless networking, wired networking, etc.), and the like.

[0188] In some examples, one or more of the cores 502(A)-(N) have multithreading capabilities. The system agent unit circuit system 510 includes those components that coordinate and operate the cores 502(A)-(N). The system agent unit circuit system 510 may include, for example, a power control unit (PCU) circuit system and / or a display unit circuit system (not shown). The PCU may be (or may include) the logic and components required to regulate the power state of the cores 502(A)-(N) and / or the dedicated logic 508 (e.g., integrated graphics logic). The display unit circuit system is used to drive one or more externally connected displays.

[0189] The cores 502(A)-(N) may be homogeneous with respect to an instruction set architecture (ISA). Alternatively, the cores 502(A)-(N) may be heterogeneous with respect to an ISA; that is, a subset of the cores 502(A)-(N) may be capable of executing an ISA, while other cores may be capable of executing only a subset of the ISA or may be capable of executing another ISA.

[0190] Example Core Architecture - In-order and Out-of-order Core Block Diagram

[0191] Fig. 6AThe block diagram of illustrates both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to an example. Figure 6B The block diagram illustrates both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor, according to an example. Figure 6A-6B The solid line boxes in illustrate the in-order pipeline and in-order core, while the optionally added dashed line boxes illustrate the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0192] exist Fig. 6A , the processor pipeline 600 includes a fetch stage 602, an optional length decode stage 604, a decode stage 606, an optional allocation (Alloc) stage 608, an optional rename stage 610, a dispatch (also called dispatch or issue) stage 612, an optional register read / memory read stage 614, an execute stage 616, a write back / memory write stage 618, an optional exception handling stage 622, and an optional commit stage 624. One or more operations may be performed in each of these processor pipeline stages. For example, during the fetch stage 602, one or more instructions are fetched from the instruction memory, and during the decode stage 606, the fetched one or more instructions may be decoded, an address using a forwarded register port (e.g., a load store unit (LSU) address) may be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) may be performed. In one example, the decode stage 606 and the register read / memory read stage 614 may be combined into one pipeline stage. In one example, during the execution stage 616, decoded instructions may be executed, LSU address / data pipelining to an Advanced Microcontroller Bus (AMB) interface may be performed, multiplication and addition operations may be performed, arithmetic operations with branch results may be performed, and the like.

[0193] As an example, Figure 6B The example register renaming, out-of-order issue / execution architecture core may implement pipeline 600 in the following manner: 1) instruction fetch circuitry 638 performs fetch and length decode stages 602 and 604; 2) decode circuitry 640 performs decode stage 606; 3) rename / allocator unit circuitry 652 performs allocate stage 608 and rename stage 610; 4) (one or more) scheduler circuit systems 656 perform the scheduling stage 612; 5) (one or more) physical register file circuit systems 658 and memory unit circuit systems 670 perform the register read / memory read stage 614; (one or more) execution clusters 660 perform the execution stage 616; 6) the memory unit circuit system 670 and (one or more) physical register file circuit systems 658 perform the write back / memory write stage 618; 7) various circuit systems may be involved in the exception handling stage 622; and 8) the retirement unit circuit system 654 and (one or more) physical register file circuit systems 658 perform the commit stage 624.

[0194] Figure 6B Processor core 690 is shown to include front end unit circuitry 630 coupled to execution engine unit circuitry 650, and both are coupled to memory unit circuitry 670. Core 690 may be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, core 690 may be a special purpose core such as, for example, a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0195] The front end unit circuit system 630 may include a branch prediction circuit system 632, which is coupled to an instruction cache circuit system 634, which is coupled to an instruction translation lookaside buffer (TLB) 636, which is coupled to an instruction fetch circuit system 638, which is coupled to a decode unit circuit system 640. In one example, the instruction cache circuit system 634 is included in the memory unit circuit system 670 instead of the front end circuit system 630. The decoding circuit system 640 (or decoder) can decode the instruction and generate one or more micro-operations, micro-code entry points, micro-instructions, other instructions or other control signals as outputs, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decoding circuit system 640 may also include an address generation unit (AGU, not shown) circuit system. In one example, the AGU uses the forwarded register port to generate the LSU address and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decode circuit system 640 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read only memory (ROM), and the like. In one example, the core 690 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in the decode circuit system 640 or otherwise within the front-end circuit system 630). In one example, the decode circuit 640 includes a micro-op or operation cache (not shown) to save / cache decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 600. The decode unit circuit system 640 may be coupled to the rename / allocator unit circuit system 652 in the execution engine unit circuit system 650.

[0196] The execution engine circuitry 650 includes a rename / allocator unit circuitry 652 coupled to a retirement unit circuitry 654 and a set of one or more scheduler circuitry 656. The scheduler circuitry 656 represents any number of different schedulers, including reservation stations, central instruction windows, and the like. In some examples, the scheduler circuitry 656 may include an arithmetic logic unit (ALU) scheduler / scheduling circuitry, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuitry, an AGU queue, and the like. The scheduler circuitry 656 is coupled to the physical register file(s) circuitry 658. Each of the physical register file(s) circuitry 658 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., instruction pointers, i.e., the address of the next instruction to be executed), and the like. In one example, the physical register file(s) circuitry 658 includes a vector register unit circuitry, a write mask register unit circuitry, and a scalar register unit circuitry. These register units may provide architectural vector registers, vector mask registers, general purpose registers, and the like. The physical register file(s) circuitry 658 is coupled to a retirement unit circuitry 654 (also referred to as a retirement queue) to demonstrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using a reorder buffer(s) (ROB) and a retirement register file(s); utilizing a future stack(s), a history buffer(s), and a retirement register file(s); utilizing a register map and a pool of registers; and the like). The retirement unit circuitry 654 and the physical register file(s) circuitry 658 are coupled to an execution cluster(s) 660. The execution cluster(s) 660 includes a set of one or more execution unit circuitry 662 and a set of one or more memory access circuitry 664. Execution unit circuitry 662 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some examples may include several execution units or execution unit circuitry dedicated to a particular function or set of functions, other examples may include only one execution unit circuitry or multiple execution units / execution unit circuitry that all perform all functions.Scheduler circuitry(s) 656, physical register file unit circuitry(s) 658, and execution cluster(s) 660 are shown as possibly multiple because some examples create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each with its own scheduler circuitry, physical register file circuitry(s), and / or execution cluster—and in the case of a separate memory access pipeline, only the execution cluster of that pipeline has memory access unit circuitry(s) 664 in some examples of the implementation). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest are in-order.

[0197] In some examples, the execution engine unit circuitry 650 may perform load store unit (LSU) address / data pipelining to an advanced microcontroller bus (AMB) interface (not shown), as well as address stage and writeback, data stage loads, stores, and branches.

[0198] A set of memory access circuitry 664 is coupled to the memory cell circuitry 670, which includes a data TLB circuitry 672, which is coupled to a data cache circuitry 674, which is coupled to a level 2 (L2) cache circuitry 676. In one example, the memory access circuitry 664 may include a load unit circuitry, a store address unit circuitry, and a store data unit circuitry, each of which is coupled to the data TLB circuitry 672 system in the memory cell circuitry 670. The instruction cache circuitry 634 is further coupled to the level 2 (L2) cache circuitry 676 in the memory cell circuitry 670. In one example, the instruction cache 634 and the data cache 674 are combined into a single instruction and data cache in the L2 cache circuitry 676, a level 3 (L3) cache circuitry (not shown), and / or a main memory (not shown). The L2 cache circuitry 676 is coupled to one or more other levels of cache and ultimately to the main memory.

[0199] Core 690 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions that have been added with newer versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instruction(s) described herein. In one example, core 690 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), allowing operations used by many multimedia applications to be performed using packed data.

[0200] Example Execution Unit(s) Circuitry

[0201] Figure 7 An example of execution unit circuitry, such as Figure 6B (One or more) execution unit circuit systems 662. As illustrated, (one or more) execution unit circuit systems 662 may include one or more ALU circuits 701, optional vector / single instruction multiple data (single instruction multiple data, SIMD) circuits 703, load / store circuits 705, and / or branch / jump circuits 707, and / or floating-point unit (FPU) circuits 709. ALU circuits 701 perform integer arithmetic and / or Boolean operations. Vector / SIMD circuits 703 perform vector / SIMD operations on packed data (such as SIMD / vector registers). Load / store circuits 705 execute load and store instructions to load data from memory into registers, or store data from registers to memory. Load / store circuits 705 may also generate addresses. Branch / jump circuits 707 cause branches or jumps to a certain memory address depending on the instruction. FPU circuits 709 perform floating-point arithmetic. The width of the execution unit circuitry 662 varies depending on the example, and may range from 16 bits to 1024 bits, for example. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0202] Program code can be applied to input information to perform the functions described herein and generate output information. Output information can be applied to one or more output devices in a known manner. For the present application, processing system includes any system with a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microprocessor, or any combination thereof.

[0203] Program code can be implemented in a process-oriented or object-oriented high-level programming language to communicate with the processing system. If desired, program code can also be implemented in assembly or machine language. In fact, the mechanism described herein is not limited to any specific programming language in scope. In any case, the language can be a compiled language or an interpreted language.

[0204] Examples of the mechanisms disclosed in this document can be implemented in hardware, software, firmware, or a combination of these implementation approaches. The examples can be implemented as computer programs or program codes, executed on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0205] One or more aspects of at least one example may be implemented by representative instructions stored on a machine-readable medium, which represent various logics within a processor, and which, when read by a machine, cause the machine to make logic for performing the techniques described herein. These representations are referred to as "intellectual property (IP) cores" and may be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities to be loaded into a manufacturing machine that makes the logic or processor.

[0206] Such machine-readable storage media may include, but are not limited to, a non-transitory tangible arrangement of an article manufactured or formed by a machine or device, including storage media such as: a hard disk, any other type of disk (including floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and magneto-optical disks), semiconductor devices (e.g., read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase change memory (PCM)), magnetic or optical cards, or any other type of medium suitable for storing electronic instructions.

[0207] Therefore, examples also include non-transitory tangible machine-readable media containing instructions or containing design data, such as hardware description languages ​​(HDL), that define the features of structures, circuits, devices, processors and / or systems described herein. Such examples may also be referred to as program products.

[0208] Emulation (including binary translation, code deformation, etc.)

[0209] In some cases, the instruction converter can be used to convert instructions from a source instruction set architecture to a target instruction set architecture. For example, the instruction converter can translate (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate or otherwise convert instructions to one or more other instructions to be processed by the core. The instruction converter can be implemented with software, hardware, firmware or a combination thereof. The instruction converter can be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0210] Figure 81 is a block diagram illustrating the use of a software instruction converter according to an example to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Figure 8 It is shown that a program in a high-level language 802 can be compiled using a first ISA compiler 804 to generate first ISA binary code 806 that can be natively executed by a processor having at least one first ISA core 816. The processor having at least one first ISA core 816 represents any processor that is capable of either (1) compatibly executing or otherwise processing a substantial portion of the first ISA or (2) executing a program in a first ISA binary format on a processor having at least one first ISA core. A target code version of an application or other software is run on a processor to execute with at least one first ISA core The first ISA compiler 804 represents a compiler operable to generate first ISA binary code 806 (e.g., object code) that can be executed on a processor 816 having at least one first ISA core with or without additional linking processing. Similarly, Figure 8 A program in the form of a high-level language 802 is shown that can be compiled using an alternative ISA compiler 808 to generate an alternative ISA binary code 810 that can be natively executed by a processor 814 without a first ISA core. An instruction converter 812 is used to convert the first ISA binary code 806 into code that can be natively executed by a processor 814 without a first ISA core. This converted code may not necessarily be the same as the alternative ISA binary code 810; however, the converted code will implement the overall operation and be composed of instructions from the alternative ISA. Thus, the instruction converter 812 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have a first ISA processor or core to execute the first ISA binary code 806 through simulation, emulation, or any other process.

[0211] References to "one example," "an example," "one embodiment," "an embodiment," etc. indicate that the described example or embodiment may include a particular feature, structure, or characteristic, but each example or embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same example or embodiment. Further, when a particular feature, structure, or characteristic is described in conjunction with an example or embodiment, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other examples or embodiments, whether or not explicitly described.

[0212] In addition, in the examples described above, unless otherwise specifically noted, delimited language such as phrases "at least one of A, B, or C" or "A, B, and / or C" is intended to be understood to mean A, B, or C, or any combination thereof (i.e., A and B, A and C, B and C, and A, B, and C). As used in this specification and claims, and unless otherwise specified, the use of ordinal adjectives "first," "second," "third," etc., used to describe elements merely indicates that a specific instance of an element or different instances of similar elements are being referenced, and is not intended to imply that the elements so described must be in a particular order in time, space, hierarchically, or in any other manner. In addition, as used in the description of the embodiments, the " / " character between items may mean that what is described may include the first item and / or the second item (and / or any other additional items), or may be implemented using, utilizing, and / or according to the first item and / or the second item (and / or any other additional items).

[0213] In addition, the terms "bit", "flag", "field", "entry", "indicator", etc. may be used to describe a storage location in a register, table, database, or other data structure of any type or content, whether implemented in hardware or software, and these terms are not intended to limit the embodiments to any particular type of storage location or the number of bits or other elements within any particular storage location. For example, the term "bit" may be used to refer to a bit position within a register and / or data stored or to be stored in the bit position. The term "clear" may be used to indicate storing a logic value of 0 in a storage location, or otherwise causing a logic value of 0 to be stored in a storage location; and the term "set" may be used to indicate storing a logic value of 1, all 1s, or some other specified value in a storage location, or otherwise causing a logic value of 1, all 1s, or some other specified value to be stored in a storage location; however, these terms are not intended to limit the embodiments to any particular logical convention, as any logical convention may be used within the embodiments.

[0214] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the claims.

Claims

1. A processing device for quality of service support, comprising: an execution circuit system for executing a plurality of software threads; hardware for controlling monitoring or allocation of one or more shared resources among the plurality of software threads; as well as Configuration storage for enabling monitoring or allocation of the one or more shared resources between the plurality of software threads and between one or more channels through which one or more devices are connected to the one or more shared resources.

2. The processing device according to claim 1, wherein: The configuration storage device is also used to associate a quality of service tag with the multiple software threads for monitoring or allocating the one or more shared resources among the multiple software threads, and wherein the one or more channels are also used to associate with a quality of service tag for monitoring or allocating the one or more shared resources among the one or more channels.

3. The processing device according to claim 2, wherein: The quality of service tag includes a resource monitoring identifier.

4. The processing device according to claim 2, wherein: The quality of service tag includes a class of service value.

5. The processing device according to claim 1, wherein: The one or more shared resources include a shared cache.

6. The processing device according to claim 1, wherein: The one or more shared resources include bandwidth to memory.

7. The processing device according to claim 1, wherein: The one or more devices include one or more input / output devices.

8. The processing device according to claim 1, wherein: The one or more devices include one or more accelerators.

9. The processing device according to claim 1, wherein: The one or more devices include a peripheral component interconnect express device.

10. The processing device according to claim 1, wherein: The one or more devices include a computing fast link device.

11. The processing device according to claim 1, wherein: The one or more channels are configured to be mapped to the one or more devices using one or more high-level configuration and power interface data structures.

12. The processing device of claim 1, wherein: The configuration storage device is further configured to associate a quality of service label with the one or more channels.

13. A method for quality of service support, comprising: enabling, by programming configuration storage in a processing device, monitoring or allocation of one or more shared resources between a plurality of software threads and between one or more channels through which one or more devices are connected to the one or more shared resources; as well as During execution of the plurality of software threads by the processing device, monitoring or allocation of the one or more shared resources between the plurality of software threads and between the one or more channels is controlled.

14. The method of claim 13, further comprising: By programming the configuration storage device in the processing device, a quality of service tag is associated with the multiple software threads for monitoring or allocating the one or more shared resources among the multiple software threads, wherein the one or more channels are also used to be associated with the quality of service tag for monitoring or allocating the one or more shared resources among the one or more channels.

15. The method of claim 14, wherein: The quality of service tag includes a resource monitoring identifier or a service category value.

16. The method of claim 13, wherein: The one or more shared resources include a shared cache or bandwidth to a memory.

17. The method of claim 13, wherein: The one or more devices include one or more input / output devices, one or more accelerators, one or more peripheral component interconnect express devices, or one or more compute express link devices.

18. The method of claim 13, further comprising: The one or more channels are mapped to the one or more devices by configuring one or more high-level configuration and power interface data structures.

19. A system for quality of service support, comprising: one or more input / output devices; as well as A processing device, the processing device comprising: an execution circuit system for executing a plurality of software threads; hardware for controlling monitoring or allocation of one or more shared resources among the plurality of software threads; and Configuration storage means for enabling monitoring or allocation of the one or more shared resources between the plurality of software threads and between one or more channels through which one or more input / output devices are connected to the one or more shared resources.

20. The system of claim 19, wherein: The one or more input / output devices include one or more peripheral component interconnect express devices or one or more compute express link devices.

Images