Automatic ai-guided processor system tuning

Abstract

Aspects of system optimization and performing tuning are described. An example method to operate an AI-guided manageability engine includes: identifying configuration settings associated with compute circuitry; performing perturbation on the configuration settings using a trained model, with the trained model being adapted to modify the configuration settings based on a type of a workload and at least one usage condition of the workload; evaluating an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; and storing a configuration to enable use of the modified configuration settings with the compute circuitry, with the configuration tracking information associated with the at least one usage condition and the type of the workload.

Description

PATENT APPLICATIONFORAUTOMATIC AI-GUIDED PROCESSOR SYSTEM TUNINGINVENTOR(S)Francesc Guim BernatViolante MoschianoEdgar Gonzalez PellicerGaspar Mora PortaTommaso ValiSatoru TagayaErich Ludwig FochtBy:OPENCHIP & SOFTWARE TECHNOLOGIES S. L.Barcelona, SpainThis invention was made with government support under Grant UNICO-IPCEI-2023-001 funded by the European Union-Next Generation EU, Important Projects of Common European Interest (IPCEI).Funded by Plan Recu the European Union ■««> TraiwforiTiaciNexOe& '4,,t nerationEUAUTOMATIC AI-GUIDED PROCESSOR SYSTEM TUNINGSTATEMENT OF FUNDING

[0001] This invention was made with government support under Grant UNICO-IPCEI-2023-001 funded by the European Union-Next Generation EU, Important Projects of Common European Interest (IPCEI).BACKGROUND

[0002] Current computing systems are often composed of large deployments of different hardware arrangements of different sizes (e.g., hardware in a 1 U rack unit, half a rack, or multiple racks) that potentially host different types of platforms and computing technologies (e.g., different processor models, different accelerators, etc.). Additionally, there is a large number of software applications and uses (e.g., databases, video analytics, content delivery, etc.) being developed by different companies that exhibit different behaviors and utilize the aforementioned resources in different ways. For example, some software applications may be more I / O centric, other software applications may be compute-centric, and even different types of workloads with the same software applications may have different effects on the processing hardware. This presents a significant challenge for system optimization.

[0003] Today, system optimization is usually performed by customer experts, sometimes together with system integrators. However, customers and integrators only have partial access to some of the system knobs that are available to them — settings that control the power, frequency, memory bandwidth, etc. For example, System-on-Chip (SoC) devices may include many other knobs, such as memory controller queue size, that only can be accessible or changed by the manufacturer itself and cannot be optimized or tuned for a particular deployment.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] In the drawings, which are not necessarily drawn to scale, reference numerals are repeated to describe similar components in different views. Like numerals having different letter suffixes may represent different instances of similar components. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document.

[0005] FIG. 1 depicts an architecture of a computing system having processor circuitry usable with the present system tuning techniques, according to an example.

[0006] FIG. 2 depicts a flowchart of an example process for initiating optimization in processing circuitry with an Al-guided manageability engine, according to an example.

[0007] FIG. 3 depicts implementation details of the Al-guided manageability engine, according to an example.

[0008] FIG. 4 depicts a further view of a computing system having processing circuitry adapted for performing and evaluating changes to a platform configuration, according to an example.

[0009] FIGS. 5A and 5B depict additional views of a computing system having processing circuitry adapted for implementation of multiple Al-guided manageability engines, according to an example.

[0010] FIG. 6 depicts a flowchart of a process for determining a compute configuration of processing circuitry based on modified configuration settings, according to an example.

[0011] FIG. 7 depicts a hardware arrangement of a data center used to provide multiple implementations or instances of a computing system, according to an example.

[0012] FIGS. 8A and 8B depict arrangements of a chip assembly with expanded views of chiplets and processing units, according to an example.

[0013] FIG. 9 depicts a block diagram of a computing system, according to an example.DETAILED DESCRIPTION

[0014] The following introduces implementations of computer hardware units and programmable operations, applicable in processor architectures such as chiplet-based processors, and various implementations of processing circuitry in a computing system. The platform hardware is adapted to include artificial intelligence (Al) guided Automatic Tuning functions that can perform platform optimizations to the compute circuitry (e.g., processor(s), accelerators, etc.) and related platform elements in an autonomous manner, including functions that provide optimizations at a large scale. To enhance this approach in a chiplet-based processor platform where multiple cores, processing units, and chiplets are coordinated together to perform distributed compute functions for the same or related workloads, the platform hardware is also adapted to provide a specialized type of management engine, in hardware, for operating and improving the Al tuning functions and tracking theresults of different internal optimizations across multiple compute elements. This management engine may be implemented within a dedicated chiplet or other processing circuitry that can perform advanced actions to explore settings and configuration changes and determine the results of tuning different system knobs for a variety of different conditions and scenarios.

[0015] In various examples, this functionality allows the operator of the compute hardware (e.g., a data center or appliance owner) to explore and test out platform setting changes in appropriate scenarios. For instance, the operator can enable the system to utilize a valley of compute demand (e.g., during low times of scheduled usage, during night-time, holidays, etc.) or other favorable circumstances to try to find a better internal configuration of hardware configuration knobs that optimize the execution of specific workload types. Consider a scenario where a solar-powered appliance has reached an irradiated energy maximum and an associated energy store state (e.g., where the batteries are full) such that the excess power will be lost if not used; or other situations a peak of renewable energy is occurring, and excess power is inexpensive. This type of scenario is very favorable for the exploration and tuning of system optimization to test out different settings and determine which combination of setting changes will improve the system performance — as measured by workload throughput, resource utilization, or some other measurement of optimization (e.g., associated with key performance indicators (KPIs)). Also, consider a scenario where time may be intentionally used at the beginning of a paid customer session to optimize the remainder of a session. The exploration and tuning of system optimization may provide benefits for system performance throughout the entire customer session.

[0016] One of the main limitations of existing approaches for system optimization is that systems are often designed to be generic, even though the system workload demand and resource usage will tend to mutate over time. Computing systems may also experience certain resource changes (e.g., the addition of accelerator resources or the change of storage devices), which may cause the configuration knob settings provided from the manufacturer or system integrator to not remain a best fit for the system. Further, the end customer may use a specific combination of different types of services that can be improved with a different configuration not envisioned by the manufacturer or system integrator. Thus, an important technical problem to be addressed is how to provide interfaces to the system management software stacks that can influence the hidden configuration knobs of the platformbased on the current snapshot of the system (e.g., type of services, hardware configuration, ambient, etc.) — without directly exposing the platform knobs to management software.

[0017] FIG. 1 depicts an example architecture of a computing system, usable with the system tuning techniques discussed herein. This architecture shows a compute platform 120 (e.g., provided from processing circuitry implemented as a SoC, System-on-Package (SoP), System-in-a-Package (SiP), or as a compartmentalized chipset with multiple chips and packages) that includes a network interface 121 to perform I / O operations (e.g., with network communication circuitry), hardware elements 123 (e.g., implemented by specialized compute circuitry such as a central processing unit (CPU), accelerator, and constituent elements such as cores, caches, controllers, etc.), and a management software stack 122 (e.g., loaded software instructions and data) that executes on the hardware elements 123. The hardware elements 123 may be implemented in the compute platform 120 as specialized compute circuitry to perform compute operations for some workload.

[0018] The compute platform 120 is also depicted as including a memory controller 150 (e.g., memory controller circuitry implemented on the same package as the hardware elements 123 or nearby the hardware elements 123). The memory controller 150 is used to write and read data from memory units 160A, 160B, 160C, such as respective memory channel modules (e.g., dual in-line memory modules (DIMMS) such as SDRAM modules). Additional implementation examples of the compute platform 120 are provided with reference to FIGS. 7, 8A, 8B, and 9, discussed below.

[0019] The compute platform 120 may be adapted to receive commands from a management software stack 110, such as an on-cloud implementation of management software. The management software stack 110 may provide in-band or out-of-band communications 112 (e.g., received via the network interface 121) that monitor the state of the hardware elements and provide commands to respond to system conditions and change settings. The management software stack 122 may collect and analyze telemetry data in response to hardware events detected within the hardware elements 123 and provide changes to hardware configuration settings based on detected conditions. For instance, the management software stack 122 may perform power management and other system-wide control functions thatchange hardware operations of the compute platform 120, using remedial actions that respond to events and conditions indicated in telemetry data.

[0020] The compute platform 120 is shown as including various changeable system settings, which are generally referred to as “knobs”, referring to the analogy of a rotating “knob” that can be actuated in either direction to increase or decrease some operational characteristic of the platform circuitry (e.g., by defining a value within some available range of values) that cause some effect and performance impact on the overall platform. Such knobs may be implemented by a control register such as a model-specific register (MSR) in an x86 processor architecture, System Control Registers (SCRs) in an ARM processor architecture, a Control and Status Register (CSR) in a RISC-V processor architecture, and the like. Some of these registers may be visible and discoverable to the management software stacks 110 or 122, such as those register settings included in visible configuration registers 131; other of the registers may not be visible and discoverable, such as shown with hidden configuration registers 132.

[0021] An Al-guided manageability engine 140, implemented in the compute platform 120 within manageability configuration circuitry, operates to provide exploration, testing, tuning, and selection of various configuration settings. These configuration settings may include values used in the visible configuration registers 131, the values used in the hidden configuration registers 132, and other registers or hardware element features of the platform — all of which are interchangeably referred to as “knobs” or “configuration knobs”. Additional details of the manageability engine 140 are provided with reference to FIG. 3 below.

[0022] The compute platform 120 can be operably connected to a variety of external devices 170, including other PCIe-based devices or peripherals in a larger computing system or deployment. The external devices 170 may expose tuning functions that can be adjusted by the manageability engine 140 or separate implementations of these manageability engines (e.g., manageability engines implemented directly within the external devices 170). Example configurations of manageability engines in external devices are discussed below with reference to FIGS. 5A and 5B.

[0023] In the context of configuring and setting platform configuration knobs on the compute platform 120, some prior approaches have involved minor optimization techniques from the management software stacks 110 or 122 based on telemetry data collection, such as fingerprinting, mile markers, dynamic control loops, etc.However, not all the configuration knobs that can be tuned in a platform are accessible to end users — such as the hidden configuration registers 132 — and thus, such knobs are not accessible by the management software stacks 110 or 122. Processors and other of the chip elements 123 may have a variety of registers that affect system operations that are not exposed for IP, security, or reliability reasons.

[0024] Many manufacturers or system integrators spend a substantial amount of time defining different combinations of platform configurations of the knobs in an effort to try to optimize how those platforms will work in different deployment models. Additionally, some performance analysis tools can be used by end customers to benchmark different types of applications executed by processor facilities, to try to find out what platform configurations and visible knob changes provide the best performance for certain types of workloads. However, existing tools do not provide methods to optimize the platform’s internal configuration without accessing and exposing all the knobs to the end customer. These limitations are overcome by the following features of the manageability engine 140, which provide a mechanism to safely explore and modify hidden configuration settings within the hardware itself.

[0025] FIG. 2 depicts a flowchart 200 of an example process for initiating optimization in processing circuitry with the manageability engine 140, implemented within manageability configuration circuitry. This process may be expanded with additional interface commands, functions, and actions of the manageability engine 140, detailed in FIG. 3 below. In the following examples, optimization refers to changes in hardware, resulting from perturbations that improve some measured aspect of how the underlying hardware or system works (e.g., with some metric that measures that the circuitry is working faster, more efficiently, with use of fewer resources, etc.). However, optimization also can be achieved and measured based on optimal importance values, measurements of completion of some standardized problem, resource utilization and / or sharing, resource demand or smoothing, sustainability considerations, or other objectives.

[0026] At operation 210, the manageability engine 140 detects a system condition for running the optimization process. This may occur during low valley periods or other system-wide conditions that favor the evaluation of system changes (e.g., excess of solar energy, excess processing, network, or memory resources, reduced disruption to other services, decrease in scheduled or forecasted usage, etc.). This system condition may also occur in specific situations where the systemencounters a need for improved configuration (such as when the usage of some resource is unexpectedly maximized for some period of time).

[0027] At operation 220, the manageability engine 140 starts an auto-exploration of the configuration settings of the computing platform, using an optimization process (e.g., provided generally by operations 230 to 260). This auto-exploration may be triggered or scheduled in response to the detection of the system condition (or multiple system conditions / pre-conditions) and / or management preferences regarding when to perform the optimization.

[0028] At operation 230, the manageability engine 140 performs a controlled perturbation of system knobs by providing changes to configuration settings. Such perturbation may include controlled variations such as a minor disturbance or irregular value to modify existing configuration settings. Such perturbation may also be informed based on the results of prior modifications, such as to increase the amount of the modifications that provided improvements and to decrease the amount of the modifications that did not provide improvements. These changes may be based on Al-based logic managed by the manageability engine 140, including reinforcement learning, which attempts to explore and try randomized amounts and types of changes to configuration settings to find new settings that produce a beneficial effect on system performance.

[0029] At operation 240, the system evaluates the impact of the perturbations on system performance, and especially application performance, based on system conditions. The current system conditions related to the type of workload, the usage condition of the workload, and measurements of KPIs may be considered to determine whether the modified configuration settings have improved or not improved (or made no change) to the performance, sustainability, or resiliency of the compute circuitry. The measurements and evaluations of KPIs may be considered as improved (or not improved) from the perspective of any of: the party requesting the workload; the system operator; customers; resource operators; and the like. As an example of system conditions, an application can report, on an ongoing basis, application metrics relevant to a particular use case of the application such as for video processing “frames per second / watt”. The application might report 10 fps / watt with a given configuration, but with a modification of the system knobs in a configuration of the memory controller, the application performance is improved to 15 fps / watt. Here, the goal always is to find the configurations that provide the mostbenefit to the underlying applications being executed, which might not necessarily correspond to resource KPIs (e.g., memory bandwidth).

[0030] As an example of perturbations, using a platform interface to the manageability engine 140, a system software stack (e.g., management software stack 110 or 122 or other software) triggers the manageability engine 140 to explore changes to some of the internal configuration knobs. The changes can be provided from an approach similar to a random forest algorithm that uses multiple decision trees to evaluate many different outcomes of the modified configuration settings. The settings to be explored can relate to multiple types of resources generally or may be specific to certain types of resources. Thus, during operations 230 and 240, the manageability engine 140 will implement and evaluate different random or guided configuration changes (e.g., modified configuration settings) to the visible and hidden knobs and compare the changes against the application KPIs or metrics that the system software stack is reporting. In particular, two types of metrics may be evaluated and optimized for the system software stack: application metrics (e.g. fps, queries per second, etc.); and resource metrics that will have an impact on the application metrics (e.g., memory bw, flops / watt, etc).

[0031] At operation 250, the manageability engine 140 creates one or more system configurations mapped to conditions and workloads. For example, the platform can internally store the identifier (e.g., the hash) in some persistent memory, associating the hash with the actual configuration settings used in the best-identified compute element configuration. Thus, the configuration will include various settings that modify both the visible configuration registers 131 and the hidden configuration registers 132.

[0032] At operation 260, the auto-exploration of optimization is completed. For example, once the management software stacks 110 or 122 provide a command to stop the optimization, the manageability engine 140 can provide the management software stack(s) with some identifier (e.g., a hash) that identifies the best-found configuration based on the achieved KPIs.

[0033] At operation 270, the configuration(s) and / or the optimization model are adapted and improved based on the impact of the configuration(s) and learnings encountered from running the optimization processes. For instance, the management software stacks 110 or 122 can use the different discovered compute element configurations over time to improve the configurations — using the identifier (hash) as a reference — and explore related platform resources (e.g., memory,accelerators) that can offer further optimization. The identifier can also be used during production times to set up the platform configuration depending on the type of service being executed.

[0034] Such techniques can be used to evaluate optimizations for executing single-service and multi-service processes or more complicated scenarios involving the execution of processes for multiple tenants or compute users. Further, these techniques may be enhanced by analysis of telemetry data of the platform in order to guide which hidden knobs are most suitable to be tested and optimized by the logic. As will be understood, these optimization techniques may not be useful for some types of workloads and services that involve large data processing phases (e.g., genomic analysis). However, these optimization techniques may be suitable for many types of applications provided in a data center that involve steady workload usage at a particular injection rate.

[0035] FIG. 3 depicts additional implementation details of the manageability engine 140, implemented in the compute platform 120 within manageability configuration circuitry, including aspects of Al processing for discovering and changing various platform, system, and device knobs. Specifically, this diagram shows new interfaces provided for the manageability engine 140 (e.g., interfaces 310 and application programming interfaces (APIs) 351, 352, 353) that can allow configuration of the optimization process. Three types of interfaces are discussed below, but it will be understood that additional or fewer interfaces can be used.

[0036] A first type of interface allows the management software stack to notify the manageability engine 140 where the application hints will be stored. For example, manageability engine 140 may be adapted to provide various MSR application APIs 351, 352, 353 to receive a list of MSR(s) that indicate where application KPIs 304 — and the corresponding Service Level Objective (SLO) goals for each of the application KPIs 304 — are going to be stored. For instance, such information may specify that MSR1 will be used to store an application metric that needs to be evaluated with the configuration (e.g., frames per second for video processing, although many other types of metrics and processing use cases may be applicable). These APIs or the interfaces 310 of the manageability engine 140 may also receive the SLO associated with the application. This SLO information can be used by the manageability engine 140 to compare system configuration modifications to the actual SLO impact.

[0037] A second type of interface enables the application or software stack (e.g., the management software stack 110 or 122) to cause the manageability engine 140 to test determined conditions and potentially find a new modified configuration. For example, the interfaces 310 may include an interface that starts the internal sweep of configuration analysis with system knob sweep logic 320, and such an interface may receive specifications of what types of resources (or a list of resources) are to be evaluated during the sweep flow. The specifications can include details on specific resources (e.g., memory channels, home agents, memory controllers, etc.) or knobs associated with specific resource types (e.g., registers and values related to memory, knobs related to I / O, etc.). The specifications may also provide a duration of the evaluation for a particular configuration, including relevant value(s), ranges of values(s), and potentially units of time (e.g., 100ms). Such information can inform the manageability engine 140 about how long a given modified configuration needs to be tested in the platform.

[0038] A third set of interfaces allows additional interaction with the optimization process. For example, the interfaces 310 may include an interface that allows the management software stack 110 or 122 to start and / or stop the sweep, and to return information that identifies the current best-discovered configuration for a usage condition and a workload type. For example, a command may be received to stop the sweep (with function 301) and provide the best-known configuration identifier (returned as a hash identifier 302) back to the management software stack 110 or 122. The interfaces 310 may also include an interface that allows the management software stack 110 or 122 to register a specific configuration to be loaded (with function 303) based on the software stack providing a hash identifier.

[0039] The perturbation of the configuration settings for visible knobs 370 and hidden knobs 360 (e.g., system-only visible knobs) is performed by an Al sweeper 331 A controlled by the sweep logic 320. The Al sweeper 331 B may be implemented by a neural network or similar type of AI / ML model or algorithm. For example, the neural network of the Al sweeper can be trained (e.g., at Al sweeper 331 B) by an Al Training engine 340 using existing algorithms such as reinforced learning and guided via optimizers. In some examples, every sweep can generate a uniquely trained Al Sweeper model that will be associated with a certain workload and will be identified by a given hash. That unique trained model can be associated with platform knob configuration settings, performance of the applications (KPIs), and also other contextual values. The performance of the application can be determinedby the sweep logic 320 based on the collection and evaluation of system telemetry data 350 and other information collected by the manageability engine 140.

[0040] The sweep logic 320 controls access to the hash-to-platform configuration table 380 and can refer to the associated configuration of the values in hidden knobs 360 and visible knobs 370. The manageability engine 140 will associate these knob values and resource types with an identifier and store this information as a platform configuration in the configuration table 380. Thus, the sweep logic 320 establishes a combination of configuration settings for the knobs and stores the combination of configuration settings in the configuration table (and in some examples, storing only some of the changed knobs). Other information may be communicated between the manageability engine 140 and the management software stacks 110, 122 via the interfaces 310, such as information on the status of registration, registration information of a service, application, or workload API, performance information associated with a service or multiple services, and related information.

[0041] In some examples, the sweep logic 320 can also monitor how the KPI measurements evolve over time and try to find the configuration(s) that maximize those KPIs. For instance, as part of the result of the sweep, multiple identifiers (hashes) could be returned to the management software stack 110 or 122 that correspond to different optimization points for different types of services. Then, the management software stack 110 or 122 can choose one of the optimizations for use (e.g., when evaluating a multi-tenant deployment). The management software stack 110, 122 may also specify that the manageability engine 140 randomizes settings in specific configuration knobs or types of knobs.

[0042] The knobs that provide configuration settings to be perturbated can be general or specific to certain types of resources and resource uses. As noted above, however, many aspects of the perturbation may be informed by the management software. For instance, the management software stack 110 or 122 may provide pointers to certain MSRs where the current service KPIs (e.g., frames per second) are going to be populated by the system software stack. Additionally, the management software stack 110 or 122 can provide service SLAs for specific KPIs (e.g., such as specifying a minimum of 60 fps for video processing use cases). The management software stack 110 or 122 may also specify how long each configuration will run.

[0043] FIG. 4 depicts another view of the compute platform 120, as adapted when performing and evaluating perturbations to a platform configuration viaconfiguration register changes. Specifically, the compute platform 120 includes many of the elements discussed above from FIG. 1, but with additional demonstration of how the manageability engine 140 is used to modify values in the configuration registers 131, 132 as a result of a perturbation on the relevant configuration settings controlled by inference logic 410. The manageability engine 140 then saves the modified values in a known configuration 420.

[0044] For example, the manageability engine 140 can use a trained model controlled by the inference logic 410 to generate a first random configuration based on the perturbation of the hidden knobs. The inference logic 410 will compare the performance of the random configuration against the application KPIs and other telemetry data that the management software stack 122 is reporting. After evaluating this performance for a provided interval of time, the manageability engine 140 can calculate an average or median of the KPI as the associated performance value. The manageability engine 140 may also evaluate the KPI in the context of a single service or a multiple-service use case.

[0045] The manageability engine 140 will generate other configurations and direct the inference logic 410 to store one or more configurations for future use, based on evaluating aspects such as: based on how sensitive the application(s) or service(s) respond to changes of particular types of knobs; based on a level of randomness; and / or based on the telemetry data that is observed by the platform. For instance, if collected telemetry data is indicating that the last-level cache (LLC), the highest level cache that is shared by the functional units on the chip, is seeing many accesses, then the logic may decide to start adjusting hidden knobs associated with the LLC. As discussed above, the inference logic 410 may implement changes to the optimization processes using a reinforced learning model (or combinations of other models) and using linear optimization to find the right perturbations to apply. In some examples, the model itself may be optimized (e.g., reduced, or transformed into a lightweight state) based on the particular type of system optimization or objective to be achieved. Example tuning algorithms that may be suitable for settings exploration but deployed in a lightweight model include: grid search; random search; a tree-structured Parzen estimator (TPE) algorithm; a covariance matrix adaptation evolution strategy (CMA-ES) based algorithm; or a gaussian process-based algorithm. These and other exploration algorithms may use a combination of randomness plus some exploration criteria to guide the settings exploration.

[0046] In further examples, the compute platform 120 can coordinate management with external devices using the manageability engine 140, including but not limited to PCIe-based devices or other peripherals. Trusted external devices can provide similar tuning functions and capabilities to be controlled by the manageability engine 140. This can enable optimization of an entire computing system or deployment, including the compute platform 120, connected external devices 170, and other related elements such as a power distribution unit (PDU) in a data center. In still further examples, distributed manageability engines can work together to co-optimize respective configurations for a given workload. For instance, in an extended setting of a system with external devices, the manageability engines can attest and trust other manageability engines within the system. Different chiplets can provide their own manageability engines that are exposed to other manageability engines within a package (e.g., SoC / SoP / SiP) or larger system.

[0047] FIGS. 5A and 5B depict respective views of processor circuitry (e.g., a processor package) in a computing system adapted for implementation of multiple Al-guided manageability engines, with each engine implemented in a separate instance of manageability configuration circuitry. FIG. 5A provides a system diagram of a package 510A that includes chiplets and an IO hub implemented as compute circuitry, with management operations coordinated with a primary / secondary model. In this scenario, one manageability engine operates as a primary (controlling or leader) engine responsible for orchestrating and monitoring the application, and the other engine(s) operates as a secondary (controlled) engine(s) that follows commands. FIG. 5B provides a system diagram of a package 510B that also includes chiplets and an IO hub implemented as compute circuitry, with management operations coordinated in a distributed fashion among Al-guided manageability engines implemented in manageability configuration circuitry. In this scenario, multiple manageability engines can work in a decoupled and decentralized fashion, to coordinate operations based on some common metric (e.g., by accessing and evaluating application KPIs).

[0048] In the view depicted in FIG. 5A, multiple manageability engines 140A, 140B, and 140C are depicted in compute chiplets 520A, 520B, and the IO hub 530. One of these manageability engines will be established as a primary (leader) manageability engine. The primary manageability engine (e.g., engine 140A) will attest and authenticate the one or more secondary manageability engines (e.g., engines 140B, 140C) in the system, such as with use of an authentication orattestation server 540. The primary manageability engine applies the monitoring of the application KPIs and orchestrates the other Al management entities. The primary manageability engine also stores the various hashes managed by the various Al management engines, to optimize a particular application or configuration. This will map each of the hashes as one “knob” for the element or subsystem that the respective management engine represents. Accordingly, the best-known configuration returned to the software stack will be defined by a set of knobs plus a set of hashes.

[0049] In the view depicted in FIG. 5B, a distributed model is applied for external monitoring, based on the evaluation of some metric such as an application KPI 522. Here, the authentication or attestation server 540 is used to attest and authenticate the various engines in the system, such as a first manageability engine 140D at a first chiplet 520C and a second manageability engine 140E at a second chiplet 520D. Once attested, then the system software stack 550 works with each of the management engines independently to track configurations. For instance, the system software stack 550 maintains a set of hashes for a particular configuration, as each hash corresponds to one of the Al management engines in the system.

[0050] FIG. 6 depicts a flowchart 600 of an example process for determining a compute configuration based on modified configuration settings, such as provided by modifications to the configuration registers 131, 132 using the manageability engine 140 discussed herein. The following process may be implemented by processing circuitry including compute circuitry and manageability configuration circuitry that implements the manageability engine 140 using the approaches discussed with reference to FIGS. 2 to 5B; however, some of the operations may be implemented, initiated, or controlled by the management software stack 110, 122 or via other management interfaces.

[0051] Operation 610 includes identifying configuration settings associated with the compute circuitry of a compute platform (e.g., compute platform 120). In an example, the configuration settings are provided in control registers hosted on the compute circuitry (e.g., registers 131 and 132), and values in the control registers modify operational characteristics of the compute circuitry. These configuration settings may include knob settings that are hidden from management software (e.g., provided by hidden configuration registers, are only accessible at trusted layers of the hardware, and not visible, exposed, or accessible outside the SoC / SiP and thus not accessible at higher layers of the software or system). The configuration settingsmay be identified for performance optimization by the trained model, based on a particular resource type associated with the configuration settings. Examples of such configuration settings include, but are not limited to speed, power, and bandwidth settings, such as: size of a memory controller queue; amount of core-to-uncore requests; speed of the various power domains; etc.

[0052] Operation 620 includes performing perturbation on the configuration settings using a trained model (e.g., an artificial intelligence model such as a neural network). This perturbation of the configuration settings may include executing the trained model to explore changes to the configuration settings. In an example, the trained model is adapted to modify the configuration settings based on a type of a workload (a workload type) and at least one usage condition of the workload. In further examples, the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload. In some examples, the perturbation of the configuration settings is initiated in response to detection of a system condition to trigger performance optimization. For instance, a system condition may be based on some combination of power management, compute utilization, memory utilization, or network utilization of the compute circuitry.

[0053] Operation 630 includes evaluating an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload. For instance, in a scenario where management software is used, the management software can provide key performance indicators (KPIs) or other metrics such as application KPIs or resource KPIs associated with the workload type, and evaluating the impact of the modified configuration settings includes evaluating these KPIs or metrics. The manageability engine 140 and any implementing manageability configuration circuitry may include a plurality of interfaces to receive these KPIs or metrics (and optionally, an identifier of a configuration associated some set of configuration settings). In some examples, the evaluation of some setting to be optimized by the engines can be expressed and measured with the use of thresholds. For example, a threshold may be used to ensure that the performance of application X in a measured frames per second (FPS) is at least within the range of between 15-20 FPS.

[0054] Operation 640 includes storing a configuration to enable use of the modified configuration settings in the processing circuitry (or in another system location). The configuration settings can provide specific effects on at least one of: performance states, power states, or energy states, of a plurality of processor cores.In some examples, the configuration settings may be specifically tuned for sustainability or resiliency outcomes, such as to optimize a performance per watt as reported by the application. Other configuration settings may optimize application KPIs (e.g., fps) or system KPIs (e.g., power consumption, reliability of resources, or degradation of resources). The manageability engine 140 may balance these by attempting to optimize the application KPIs, while maintaining a list of system KPI targets configured internally or provided by the system software stack (e.g. an o rch estrato r).

[0055] Operation 650 includes using the configuration for subsequent execution of other workloads. This may include selecting and communicating the configuration (of the modified configuration settings) to be deployed on the same or different hardware, such as during the execution of subsequent workloads. In addition to performance objectives or requirements, the selection and deployment of a particular configuration may be based on sustainability and reliability considerations and metrics.

[0056] Further example deployments may include coordination of multiple manageability engines such as depicted in FIGS. 5A and 5B. This may include performing operations 610 to 640 at a first manageability engine that directly controls or directs one or more other manageability engines (e.g., implemented by other instances of manageability configuration circuitry). In some scenarios, the first manageability engine and the one or more other manageability engines can exist in a trusted relationship, such as based on attestation or authentication (e.g., by an authentication or attestation server 540). In other scenarios, the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on some metric (e.g., key performance indicators) provided by the system software.

[0057] FIGS. 7, 8A, 8B, and 9 respectively depict simplified aspects of example computing architectures in which any of the techniques and configurations above may be implemented. It will be understood that the elements described above for system tuning and optimization exploration may be integrated for use with various forms of the following hardware components, including for determining modified configuration settings with a variety of the following elements.

[0058] FIG. 7 depicts an example hardware arrangement of a data center 700 used to provide multiple implementations or instances of a computing system (e.g., computing system 900, discussed below), with each instance of the computingsystem being identified as a respective platform (e.g., platform 730). The data center 700 includes data center infrastructure 701, a data center network fabric 702, and a power distribution unit 703 to support multiple racks of compute platforms, with a single instance of a rack 710 depicted in FIG. 6. The data center infrastructure 701 may provide physical components that host the compute platform hardware, storage components, and networking equipment; the data center network fabric 702 may include switches and networking components to support data flows among various compute platforms and storage devices throughout the data center; and the power distribution unit 703 may include components to distribute and control power among the various compute platforms, networking, and storage devices.

[0059] The rack 710 includes but is not limited to cooling infrastructure 711, a network interface 712, and related physical components (not shown) to support discrete instances of multiple chassis. The rack 710 provides power, connectivity, and cooling to each of the multiple chassis in a single rack, with a single instance of a chassis 720 depicted in FIG. 7. The chassis 720 includes but is not limited to cooling infrastructure 721, a chassis network fabric 722, and a power supply 723, which provides cooling, network connectivity, and power to multiple platforms within the chassis, with a single instance of a platform 730 depicted in FIG. 7. It will be understood that a common data center rack configuration may include dozens of chassis, with each chassis adapted to support a number of platforms depending on the physical size of the platform hardware and supporting equipment.

[0060] The platform 730 in some implementations may be referred to as a server or node, depending on the use case for the platform 730 and the data center 700. The platform 730 includes but is not limited to implementations of a discrete computing system hosted on a single board. The platform 730 is depicted as hosting a chip assembly 740Aand chip assembly 740B on a first board provided by a printed circuitry board (PCB) or other platform board, shown as PCB 731. In some examples, the platform 730 may include only one chip package, whereas the PCB 731 depicts interconnection of multiple chip assemblies via a device-to-device interface (e.g., a PCI express (PCIe) or compute express link (CXL) interface).Additional chip packages and components (not shown) may also be hosted on the PCB 731.

[0061] Some implementations of the chip assembly 740A and 740B may be termed as a System-on-Chip (SoC) package, as modular chiplets that perform different functions are integrated into a single package — even though this chippackage is composed of multiple dies unlike a traditional SoC design that uses a single die. Other implementations of the chip assembly 740A and 740B may be termed as a System-on- Package (SoP), System-in-a-Package (SiP), or similar references to a single chip package. Various combinations of 2D, 2.5D, and 3D packaging technologies may be used to manufacture and assemble the chip package and its underlying structure, and different manufacturing processes may be used to provide chiplets and components from different process nodes (e.g., semiconductor fabrication systems).

[0062] The chip assembly 740A and chip assembly 740B are each packages that include multiple chiplets or dies for respective functions, such as separate chiplets for processing (e.g., CPU or GPU chiplets), memory (e.g., cache or high-bandwidth memory chiplets), I / O (e.g., I / O chiplets), acceleration (e.g., AI / ML acceleration chiplets), signal processing (e.g., audio or video processing chiplets), and the like. A close-up of chip assembly 740A is depicted as including a I / O Hub chiplet 741, chiplets 742, and a power supply 743. These components may be hosted on an interposer that is designed to connect multiple dies or components within a single semiconductor package (e.g., a chip package). In some examples, the chiplets 742 may be manufactured and sourced separately and later assembled into the chip package to create the chip assembly 740A. Various connections may be provided among the chiplets 742 such as with the use of Universal Chiplet Interconnect Express (UCIe) or similar chiplet-to-chiplet interfaces and interconnects (e.g.Advanced Interface Bus (AIB), Bunch of Wires (BoW), etc.), or between chiplets and on-chip memory (e.g., high-bandwidth memory (HBM)) using HBM3 (JEDEC), Universal Memory Interface (UMI), or other memory interfaces. Similar interfaces and interconnects may be used for chip-to-chip or die-to-die communications (e.g., using NVIDIA® NVLink-C2C, Cache Coherent Interconnect for Accelerators (CIX), Compute Express Link (CXL), Advanced extensible Interface (AXI), and certain implementations of PCIe, CXL, etc.).

[0063] FIG. 8A depicts an example arrangement of a chip assembly 840A (e.g., a multi-processing core implementation of chip assembly 740Aor 740B), with expanded views of the chiplets and processing units included therein. This arrangement shows how the chip assembly 840A, which may constitute a SoC, SoP, SiP, or other type of chip package, is composed from chiplets such as chiplet 810A, chiplet 81 OB, etc. and associated on-package memory (e.g., high-speed memory) such as 3D-stacked, HBM instances shown as HBM 820A, HBM 820B, interfaces(e.g., UCIe interfaces) shown as UCIe 821 A, UCIe 821 B, and I / O hub 830 (e.g., which may be implemented by a I / O chiplet). Other hardware elements of a chip package are not depicted for simplicity.

[0064] Each chiplet includes multiple processing units and each processing unit includes one or multiple cores. For instance, chiplet 810Aas depicted includes four processing units (processing unit 800A, processing unit 800B, processing unit 800C, and processing unit 800D) and an L3 cache 804. Each processing unit may include one or multiple processing cores, one or multiple caches, and optionally other processing units or elements. For instance, processing unit 800A is depicted as including two cores (core 801 A and core 801 B), vector processing unit 802, and an L2 cache 803. Accordingly, a single-core processing unit arrangement can provide 4 cores per chiplet and 8 total cores in a two-chiplet chip assembly, whereas a dualcore processing unit arrangement can provide 8 cores per chiplet and 16 total cores in a two-chiplet chip assembly. Other permutations may also be provided. A variety of signaling interfaces and protocols (not shown) may be used for core-to-core and inter-processor communications, including but not limited to the use of coherency protocols, mesh, ring, or hybrid ring-mesh interconnects, Network-on-Chip (NoC) and packet switched communications, and the like.

[0065] FIG. 8B depicts an example arrangement of a chip assembly 840B (e.g., a multi-chiplet high-performance computing (HPC) implementation of chip assembly 740A, 740B), adapted for HPC applications (e.g., parallel processing operations involving thousands, millions, or more of processors or cores operating simultaneously). The example chip assembly 840B depicts placement as a SiP, SoC, or other package onto a platform board (e.g., PCB 731), and optionally in a data center (e.g., data center 700) or in a standalone deployment setting (e.g., in a standalone computer system, mobile computing device, autonomous device, etc.).

[0066] The chip assembly 840B is composed of multiple chiplets, shown with four chiplets - chiplet 810C, chiplet 810D, chiplet 810E, and chiplet 81 OF. Each chiplet includes multiple processing units, such as 32 processing units with a corresponding L3 cache for each processing unit. Each processing unit may include one or multiple cores, such as a single-core processing unit 800E shown as part of chiplet 810C. The chip assembly 840B is also composed of corresponding memory resources, such as HBM elements corresponding to respective banks of processing units (e.g., HBM 820B and HBM 820C corresponding respective sets of processing units of chiplet 810C), UCIe interfaces, and an IO Hub.

[0067] The chip assembly and related products or devices described herein may be configured in a variety of computing system implementations. Such implementations include machine-readable non-transitory media storing machine-readable instructions and one or more processors coupled to the memory, such that executing the machine-readable instructions configure the computing system and implementing hardware (e.g., the processing unit 800, chiplet 810, chip 740, platform 730) to perform steps and operations described above for electronic systems or devices (e.g., to receive and perform system tuning operations using a manageability engine 140, etc.). It should be further understood that software including one or more computer-executable instructions that facilitate processing and operations as described above may be distributed, installed, or otherwise provided with networked devices (e.g., servers or cloud computing systems).Alternatively, in some examples, the software may be obtained and loaded (or, re-loaded / upgraded) from one or more servers and / or cloud computing systems, such as software stored on a server for distribution over the Internet, for example.

[0068] FIG. 9 depicts a block diagram of an example computing system 900 (e.g., device, apparatus, machine, etc.) that may be programmed into a special purpose machine suitable for implementing one or more embodiments for system tuning and like optimization aspects disclosed herein. For instance, the compute circuitry, manageability configuration circuitry, or other compute sub-components described above may be embodied by the computing system 900, such as in the form of a computer or specialized electronic device that includes sufficient processing power, memory resources, and communications throughput capability to perform operations consistent with the examples herein.

[0069] The computing system 900 may include at least one hardware processing unit 902 such as a central processing unit (CPU), a graphics processing unit (GPU), a vector processing unit (VPU), a neural processing unit (NPU), a hardware accelerator, or combinations or variants thereof. The at least one hardware processing unit 902 is an implementation of processor circuitry and may be embodied by various types of chip assemblies, products, or packages as discussed with reference to FIGS. 7 to 8B. Circuitry (e.g., processing circuitry) as used herein is a collection of circuits implemented in tangible entities of the computing system 900 that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time. Circuitries include members that may, alone or in combination, perform specified operations when operating. In some examples,hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired).

[0070] In an example, the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a machine-readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the machine-readable medium elements can be part of the circuitry or communicatively coupled to the other components of the circuitry when the device is operating. Also, in some examples, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time.

[0071] The computing system 900 may also include at least one memory device 904 such as volatile memory 906 and non-volatile memory 908, and at least one storage device such as removable storage 910 and / or non-removable storage 912 such as a drive unit, some or all of which may communicate with each other via an interconnect, fabric, link, or bus 920.

[0072] The computing system 900 may include an output interface 916 such as an interface connected to a display device, and an input interface 914 such as an interface connected to an alphanumeric input device or a user interface (Ul) navigation device. In some examples, a connected I / O device may also include a display device, alphanumeric input device, and navigation device that is integrated into a single unit such as a touch screen display.

[0073] The computing system 900 may additionally include a communication interface 918, such as for connection with a network interface device used to transmit and receive electronic signals on a network. The computing system 900 may also include other interfaces or hardware (not shown) in connection with a signal generation device (e.g., an audio or radio signal generation device), an output controller (e.g., for connection with a serial, universal serial bus (USB), parallel, orother wired or wireless connection such as which uses via infrared (IR) or near field communication (NFC) technologies), an input controller (e.g., for connection with sensors or peripheral devices), and the like.

[0074] Any of the memory or storage devices such as the volatile memory 906, the non-volatile memory 908, the removable storage 910, or the non-removable storage 912 may provide a machine-readable medium. Some examples of a machine-readable medium are a non-transitory medium that hosts or stores one or more sets of data structures or instructions (e.g., software instructions) embodying or utilized by any one or more of the techniques or functions described herein. Such instructions are collectively labeled as instructions 924 with respective implementations of instructions 924A, 924B, 924C, 924D, and 924E.

[0075] The instructions 924 may reside, during execution or other operation of the computing system 900, completely or at least partially within the volatile memory 906 as instructions 924B, within non-volatile memory 908 as instructions 924C, within removable storage as instructions 924D, within non-removable storage as instructions 924E, or within the hardware processing unit 902 as instructions 924A. Thus, any combination of the hardware processing unit 902, the volatile memory 906, the non-volatile memory 908, or a storage device of the removable storage 910 or non-removable storage 912 may constitute a machine-readable medium or media. The instructions 924A, when loaded and executed by the hardware processing unit 902, may invoke or utilize a defined instruction set 922 of the hardware processing unit 902, such as a processor instruction set defined by an instruction set architecture (ISA) of a reduced instruction set computer (RISC) or complex instruction set computer (CISC) architecture — including but not limited to the RISC-V Instruction Set provided in a RISC-V architecture. It will be understood that a RISC-V architecture and instruction set is one of several available architectures and instruction sets that may be used in implementations of the functional compute components (e.g., the hardware processing unit 902) discussed herein.

[0076] The term “machine-readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by components or the whole of the computing system 900 (or a similar machine) and that cause the computing system 900 or its components to perform any one or more of the techniques or functions described herein, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Non-limitingmachine-readable medium examples may include solid-state memories, and optical and magnetic media. Specific examples of machine-readable media may include non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; and optical or magneto-optical disks.

[0077] The instructions 924 may further be transmitted or received over a communications network using a transmission medium via the communication interface 918 and related devices utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as WiFi®), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others.

[0078] Method examples or other operations described herein can be implemented in part or in whole by the aforementioned machines, platforms, or devices, or related systems (including computer, robotic, and autonomous systems). The components of the illustrative devices, systems, and methods employed may be implemented in various examples by digital electronic circuitry, analog electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. These components may be implemented, for example, as a computing program product such as a computing program, program code or computer instructions tangibly embodied in an information carrier, or in a machine-readable storage device, for execution by, or to control the operation of, a data processing apparatus such as a programmable processor, a computer, or multiple computers.

[0079] A computing program may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. Also, functional programs, codes, and code segments for accomplishing the techniques described herein may be easily construed as within the scope of the present disclosure by programmers skilled in the art.

[0080] Method steps associated with the illustrative embodiments may be performed by processing circuitry executing a computing program, code, or instructions to perform operations or functions (e.g., by operating on input data and / or generating an output). Further, such operations or functions may be embodied by a machine-readable medium, which is capable of storing instructions for execution by processing circuitry (including the specific processing unit examples discussed herein), such that the instructions, when executed by the processing circuitry, cause the processing circuitry to perform any one or more of the methodologies described herein.

[0081] Computer-readable instructions can be provided as processor instructions, interpreter instructions, or other types of directives, prompts, scripts, macros, templates, code injection annotations, or other data that are directly executed (e.g., on hardware, in an interpreter, virtual machine, etc.) compiled, assembled, combined, interpreted, obfuscated, compressed, encrypted, transpiled, or modified before the execution by processing circuitry. The computer-readable instructions may be decrypted, uncompressed, unpacked, or adapted prior to execution. Thus, computer-readable instructions encompass information that is provided in executable form (e.g., object code or binary executable code), information that is used to create an executable form of code, or information that is used to derive or create intermediate information used in connection with creation, distribution, or the execution of code. Computer-readable instructions may be provided not just from a single medium or computer system, but from multiple sources including remote networked sources.

[0082] Additional examples of the presently described embodiments include the following, non-limiting implementations. Each of the following non-limiting examples may stand on its own or may be combined in any permutation or combination with any one or more of the other examples provided below or throughout the present disclosure.

[0083] Example 1 is a processing circuitry, comprising: compute circuitry, the compute circuitry comprising hardware elements to perform compute operations for a workload; and manageability configuration circuitry, the manageability configuration circuitry configured to operate a manageability engine to: identify configuration settings associated with the compute circuitry; perform perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of a workload and atleast one usage condition of the workload; evaluate an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; and store a configuration to enable use of the modified configuration settings in the processing circuitry, the configuration associated with the at least one usage condition and the type of the workload.

[0084] In Example 2, the subject matter of Example 1 optionally includes subject matter where the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

[0085] In Example 3, the subject matter of Example 2 optionally includes subject matter where the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

[0086] In Example 4, the subject matter of any one or more of Examples 1-3 optionally include subject matter where the manageability configuration circuitry is further configured to: identify a system condition to trigger performance optimization; and initiate the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

[0087] In Example 5, the subject matter of Example 4 optionally includes subject matter where the system condition to trigger performance optimization is based on power management, compute utilization, memory utilization, or network utilization of the hardware elements.

[0088] In Example 6, the subject matter of any one or more of Examples 1-5 optionally include subject matter where the configuration settings comprise settings that are provided by hidden configuration registers or are otherwise hidden from management software executing on the processing circuitry.

[0089] In Example 7, the subject matter of Example 6 optionally includes subject matter where the management software is used to provide key performance indicators (KPIs) associated with the type of the workload to the manageability configuration circuitry, and wherein operations to evaluate the impact of the modified configuration settings includes operations to evaluate the KPIs.

[0090] In Example 8, the subject matter of Example 7 optionally includes subject matter where the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

[0091] In Example 9, the subject matter of any one or more of Examples 1-8 optionally include subject matter where the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the processing circuitry that is associated with the configuration settings.

[0092] In Example 10, the subject matter of Example 9 optionally includes subject matter where to perform the perturbation of the configuration settings includes to execute the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

[0093] In Example 11, the subject matter of any one or more of Examples 1 -10 optionally include subject matter where the manageability configuration circuitry is further configured to: select the configuration to deploy the modified configuration settings for use in the processing circuitry for execution of subsequent workloads.

[0094] In Example 12, the subject matter of any one or more of Examples 1 -11 optionally include subject matter where the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

[0095] In Example 13, the subject matter of Example 12 optionally includes subject matter where the one or more other manageability engines are directly controlled by the first manageability engine.

[0096] In Example 14, the subject matter of any one or more of Examples 12-13 optionally include subject matter where the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

[0097] Example 15 is a method performed by manageability configuration circuitry, comprising: identifying configuration settings associated with compute circuitry; performing perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of a workload and at least one usage condition of the workload; evaluating an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; and storing a configuration to enable use of the modified configuration settings with the compute circuitry, the configuration associated with the at least one usage condition and the type of the workload.

[0098] In Example 16, the subject matter of Example 15 optionally includes subject matter where the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

[0099] In Example 17, the subject matter of Example 16 optionally includes subject matter where the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

[0100] In Example 18, the subject matter of any one or more of Examples 15-17 optionally include identifying a system condition to trigger performance optimization; and initiating the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

[0101] In Example 19, the subject matter of Example 18 optionally includes subject matter where the system condition to trigger performance optimization is based on power management, compute utilization, memory utilization, or network utilization of the compute circuitry.

[0102] In Example 20, the subject matter of any one or more of Examples 15-19 optionally include subject matter where the configuration settings comprise settings that are provided by hidden configuration registers or are otherwise hidden from management software.

[0103] In Example 21, the subject matter of Example 20 optionally includes subject matter where the management software is used to provide key performance indicators (KPIs) associated with the type of the workload, and wherein evaluating the impact of the modified configuration settings includes evaluating the KPIs.

[0104] In Example 22, the subject matter of Example 21 optionally includes subject matter where the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

[0105] In Example 23, the subject matter of any one or more of Examples 15-22 optionally include subject matter where the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the manageability configuration circuitry that is associated with the configuration settings.

[0106] In Example 24, the subject matter of Example 23 optionally includes subject matter where performing the perturbation of the configuration settingsincludes executing the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

[0107] In Example 25, the subject matter of any one or more of Examples 15-24 optionally include selecting the configuration to deploy the modified configuration settings for execution of subsequent workloads.

[0108] In Example 26, the subject matter of any one or more of Examples 15-25 optionally include subject matter where the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

[0109] In Example 27, the subject matter of Example 26 optionally includes subject matter where the one or more other manageability engines are directly controlled by the first manageability engine.

[0110] In Example 28, the subject matter of any one or more of Examples 26-27 optionally include subject matter where the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

[0111] In Example 29, the subject matter of any one or more of Examples 15-28 optionally include subject matter where the configuration settings provide effects on at least one of: performance states, power states, or energy states, of a plurality of processor cores.

[0112] Example 30 is at least one non-transitory machine-readable medium comprising instructions stored thereon, which when executed by manageability configuration circuitry, causes the manageability configuration circuitry to: identify configuration settings associated with compute circuitry; perform perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of a workload and at least one usage condition of the workload; evaluate an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; and store a configuration to enable use of the modified configuration settings with the compute circuitry, the configuration associated with the at least one usage condition and the type of the workload.

[0113] In Example 31, the subject matter of Example 30 optionally includes subject matter where the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

[0114] In Example 32, the subject matter of Example 31 optionally includes subject matter where the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

[0115] In Example 33, the subject matter of any one or more of Examples 30-32 optionally include subject matter where the instructions cause the manageability configuration circuitry to: identify a system condition to trigger performance optimization; and initiate the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

[0116] In Example 34, the subject matter of Example 33 optionally includes subject matter where the system condition to trigger performance optimization is based on power management, compute utilization, memory utilization, or network utilization of the compute circuitry.

[0117] In Example 35, the subject matter of any one or more of Examples 30-34 optionally include subject matter where the configuration settings comprise settings that are provided by hidden configuration registers or are otherwise hidden from management software.

[0118] In Example 36, the subject matter of Example 35 optionally includes subject matter where the management software is used to provide key performance indicators (KPIs) associated with the type of the workload, and wherein evaluating the impact of the modified configuration settings includes evaluating the KPIs.

[0119] In Example 37, the subject matter of Example 36 optionally includes subject matter where the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

[0120] In Example 38, the subject matter of any one or more of Examples 30-37 optionally include subject matter where the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the manageability configuration circuitry that is associated with the configuration settings.

[0121] In Example 39, the subject matter of Example 38 optionally includes subject matter where performing the perturbation of the configuration settingsincludes executing the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

[0122] In Example 40, the subject matter of any one or more of Examples 30-39 optionally include subject matter where the instructions cause the manageability configuration circuitry to: selecting the configuration to deploy the modified configuration settings for execution of subsequent workloads.

[0123] In Example 41, the subject matter of any one or more of Examples 30-40 optionally include subject matter where the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

[0124] In Example 42, the subject matter of Example 41 optionally includes subject matter where the one or more other manageability engines are directly controlled by the first manageability engine.

[0125] In Example 43, the subject matter of any one or more of Examples 41-42 optionally include subject matter where the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

[0126] In Example 44, the subject matter of any one or more of Examples 30-43 optionally include subject matter where the configuration settings provide effects on at least one of: performance states, power states, or energy states, of a plurality of processor cores.

[0127] In Example 45, the subject matter of any one or more of Examples 1-44 optionally include subject matter where the hardware elements include a plurality of processor cores, and wherein the configuration settings provide effects on at least one of: performance states, power states, or energy states, of the plurality of processor cores.

[0128] In Example 46, the subject matter of Example 45 optionally includes subject matter where the processing circuitry is a multi-chiplet package, wherein the compute circuitry includes at least one processor chiplet, and wherein the manageability configuration circuitry includes a chiplet separate from the at least one processor chiplet.

[0129] Example 47 is a method for system tuning, comprising operations to configure the processing circuitry according to any of Examples 1 to 46.

[0130] Example 48 is a machine-readable medium including instructions, which when executed by processing circuitry, configures the processing circuitry according to any of Examples 1 to 46.

[0131] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which the invention can be practiced. These embodiments are also referred to herein as “examples.” Such examples can include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein. In the event of inconsistent usages between this document and any documents so incorporated by reference, the usage in this document controls.

Claims

1. CLAIMSWhat is claimed is:

1. Processing circuitry, comprising:compute circuitry, the compute circuitry comprising hardware elements to perform compute operations for a workload; andmanageability configuration circuitry, the manageability configuration circuitry configured to operate a manageability engine to:identify configuration settings associated with the compute circuitry; perform perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of the workload and at least one usage condition of the workload;evaluate an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; andstore a configuration to enable use of the modified configuration settings in the processing circuitry, the configuration associated with the at least one usage condition and the type of the workload.

2. The processing circuitry of claim 1, wherein the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

3. The processing circuitry of claim 2, wherein the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

4. The processing circuitry of any of claims 1 to 3, wherein the manageability configuration circuitry is further configured to:identify a system condition to trigger performance optimization; andinitiate the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

5. The processing circuitry of claim 4, wherein the system condition to trigger performance optimization is based on at least one of power management, compute utilization, memory utilization, or network utilization of the hardware elements.

6. The processing circuitry of any of claims 1 to 5, wherein the configuration settings comprise settings provided by hidden configuration registers of the processing circuitry.

7. The processing circuitry of claim 6, wherein the management software is used to provide key performance indicators (KPIs) associated with the type of the workload to the manageability configuration circuitry, and wherein operations to evaluate the impact of the modified configuration settings includes operations to evaluate the KPIs.

8. The processing circuitry of claim 7, wherein the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

9. The processing circuitry of any of claims 1 to 8, wherein the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the processing circuitry that is associated with the configuration settings.

10. The processing circuitry of claim 9, wherein to perform the perturbation of the configuration settings includes to execute the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

11. The processing circuitry of any of claims 1 to 10, wherein the manageability configuration circuitry is further configured to:select the configuration to deploy the modified configuration settings for use in the processing circuitry for execution of subsequent workloads.

12. The processing circuitry of any of claims 1 to 11, wherein the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

13. The processing circuitry of claim 12, wherein the one or more other manageability engines are directly controlled by the first manageability engine.

14. The processing circuitry of claim 12, wherein the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

15. The processing circuitry of any of claims 1 to 14, wherein the hardware elements include a plurality of processor cores, and wherein the configuration settings provide effects on at least one of: performance states, power states, or energy states, of at least one of the plurality of processor cores.

16. The processing circuitry of claim 15, wherein the processing circuitry is a multi-chiplet package, wherein the compute circuitry includes at least one processor chiplet, and wherein the manageability configuration circuitry includes a chiplet separate from the at least one processor chiplet.

17. A method performed by a manageability engine within manageability configuration circuitry, comprising:identifying configuration settings associated with compute circuitry; performing perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of a workload and at least one usage condition of the workload;evaluating an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; andstoring a configuration to enable use of the modified configuration settings with the compute circuitry, the configuration associated with the at least one usage condition and the type of the workload.

18. The method of claim 17, wherein the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

19. The method of claim 18, wherein the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

20. The method of any of claims 17 to 19, further comprising:identifying a system condition to trigger performance optimization; and initiating the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

21. The method of claim 20, wherein the system condition to trigger performance optimization is based on at least one of power management, compute utilization, memory utilization, or network utilization of the compute circuitry.

22. The method of any of claims 17 to 21, wherein the configuration settings comprise settings provided by hidden configuration registers.

23. The method of claim 22, wherein the management software is used to provide key performance indicators (KPIs) associated with the workload type, and wherein evaluating the impact of the modified configuration settings includes evaluating the KPIs.

24. The method of claim 23, wherein the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

25. The method of any of claims 17 to 24, wherein the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the manageability configuration circuitry that is associated with the configuration settings.

26. The method of claim 25, wherein performing the perturbation of the configuration settings includes executing the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

27. The method of any of claims 17 to 26, further comprising:selecting the configuration to deploy the modified configuration settings for execution of subsequent workloads.

28. The method of any of claims 17 to 27, wherein the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

29. The method of claim 28, wherein the one or more other manageability engines are directly controlled by the first manageability engine.

30. The method of claim 28, wherein the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

31. The method of any of claims 17 to 20, wherein the configuration settings provide effects on at least one of: performance states, power states, or energy states, of a plurality of processor cores.

32. At least one non-transitory machine-readable medium comprising instructions stored thereon, which when executed by manageability configuration circuitry, causes the manageability configuration circuitry to operate a manageability engine to:identify configuration settings associated with compute circuitry;perform perturbation on the configuration settings using a trained model, wherein the trained model is adapted to modify the configuration settings based on a type of a workload and at least one usage condition of the workload;evaluate an impact of the modified configuration settings on performance of the compute circuitry during execution of the workload; andstore a configuration to enable use of the modified configuration settings with the compute circuitry, the configuration associated with the at least one usage condition and the type of the workload.

33. The at least one non-transitory machine-readable medium of claim 33, wherein the configuration settings are provided in control registers hosted on the compute circuitry, and wherein values in the control registers modify operational characteristics of the compute circuitry.

34. The at least one non-transitory machine-readable medium of claim 33, wherein the trained model is trained with reinforcement learning based on the performance of the compute circuitry during execution of the workload.

35. The at least one non-transitory machine-readable medium of any of claims 32 to 34, wherein the instructions cause the manageability configuration circuitry to:identify a system condition to trigger performance optimization; andinitiate the perturbation of the configuration settings in response to detection of the system condition to trigger performance optimization.

36. The at least one non-transitory machine-readable medium of claim 35, wherein the system condition to trigger performance optimization is based on at least one of power management, compute utilization, memory utilization, or network utilization of the compute circuitry.

37. The at least one non-transitory machine-readable medium of any of claims 32 to 36, wherein the configuration settings comprise settings that are provided by hidden configuration registers.

38. The at least one non-transitory machine-readable medium of claim 37, wherein the management software is used to provide key performance indicators (KPIs) associated with the type of the workload, and wherein evaluating the impact of the modified configuration settings includes evaluating the KPIs.

39. The at least one non-transitory machine-readable medium of claim 38, wherein the manageability configuration circuitry includes a plurality of interfaces, including at least one of an interface to receive the KPIs or an interface to communicate an identifier of the configuration.

40. The at least one non-transitory machine-readable medium of any of claims 32 to 39, wherein the configuration settings are identified for performance optimization by the trained model, based on a particular resource type of the manageability configuration circuitry that is associated with the configuration settings.

41. The at least one non-transitory machine-readable medium of claim 40, wherein performing the perturbation of the configuration settings includes executing the trained model on the manageability configuration circuitry to explore changes to the configuration settings.

42. The at least one non-transitory machine-readable medium of any of claims 32 to 41, wherein the instructions cause the manageability configuration circuitry to:selecting the configuration to deploy the modified configuration settings for execution of subsequent workloads.

43. The at least one non-transitory machine-readable medium of any of claims 32 to 42, wherein the manageability configuration circuitry is configured to implement a first manageability engine, wherein the first manageability engine is coordinated with one or more other manageability engines implemented by other instances of manageability configuration circuitry, and wherein the first manageability engine and the one or more other manageability engines exist in a trusted relationship.

44. The at least one non-transitory machine-readable medium of claim 43, wherein the one or more other manageability engines are directly controlled by the first manageability engine.

45. The at least one non-transitory machine-readable medium of claim 43, wherein the first manageability engine and the one or more other manageability engines are indirectly controlled by system software based on key performance indicators provided by the system software.

46. The at least one non-transitory machine-readable medium of any of claims 32 to 45, wherein the configuration settings provide effects on at least one of: performance states, power states, or energy states, of a plurality of processor cores.

Images