Machine Learning Power Manager for Workload Classification and Power Management Optimization

Abstract

The present invention provides a method and system, including a computer-readable medium, for running a power manager that leverages machine learning for workload classification and fine-grained power management optimization. [Solution] A computing system including a power manager acquires multiple inputs generated by system-on-chip performance counters coupled to the power manager, determines the workload type by processing the inputs based on a clustering algorithm applied to the inputs using the power manager's ML classifier model, identifies power hints based on the workload type using a data structure that maps the workload type to the power hint, and determines the power and performance policy settings for the system-on-chip processor block based on the identified power hints for the determined workload type.

Description

Technical Field

[0001] This specification generally relates to power management of integrated circuits.

Background Art

[0002] Modern computing systems often incorporate a variety of computing processing units, each offering different computing capabilities and trade-offs. Efficient execution of a computing workload often involves partitioning the computation into important tasks or subtasks that are mapped to the available processor cores of the computing system. The computation can be analyzed and mapped based on suitability criteria such as processor power, performance, and power. Generally, this overall process of allocating portions of the computation to appropriate processor resources is referred to as heterogeneous computing.

[0003] At least one processor core of a computing system can be an intellectual property block ("IP block") that executes each part of the computing operations for different multimedia use cases. Exemplary multimedia use cases can include processing image and video data captured by a multi-camera system of a mobile device such as a smartphone. The system-on-chip ("SoC") of the device can use heterogeneous computing operations to process input samples derived from the image / video data. Exemplary steps of the heterogeneous operations can include processing the image / video data to generate a corresponding output such as an image processing output that fuses content from at least two images, and providing each, any output and associated input samples to either an IP block or a machine learning engine.

[0004] In relation to machine learning models ("ML"), a neural network uses one or more layers of nodes to generate an output, such as a classification, for an incoming input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to one or more other layers in the network, such as other hidden or output layers in the network. The layers of the network can generate an output from an incoming input according to the current values ​​of their respective sets of parameters. Some neural networks may be convolutional neural networks (CNNs) configured for image and / or language processing, while others may have a transformer network architecture based on an attention mechanism, which can extend the convolutional network. [Overview of the project]

[0005] This specification describes techniques for running a power manager on a System of Chips (SoC) containing one or more hardware integrated circuits that leverages machine learning ("ML") algorithms to perform workload classification and fine-grained power management optimization. The ML power manager can run as a kernel module of the operating system kernel that applies specific global power policies to enhance the power and performance allocation of the SoC. The power manager includes a proactive ML-based workload classifier that determines workload types by collecting low-level system indices and classifying one or more existing workloads using a pre-trained ML classifier model. The ML power manager also includes a reinforcement learning model that adaptively determines and applies power policies based on the determined workload type(s).

[0006] One aspect of the subject matter described herein can be embodied in a method performed using a power manager that includes one or more neural network machine learning (ML) models. The method includes: obtaining a plurality of inputs generated by system-on-chip performance counters coupled to the power manager; determining a workload type by the power manager's ML classifier model in response to processing the plurality of inputs based on a clustering algorithm applied to the plurality of inputs by the ML classifier model; identifying power hints based on the workload types using a data structure that maps the plurality of workload types to a plurality of power hints; and determining power and performance policy settings for the system-on-chip processor block for the power hints.

[0007] These and other embodiments may each optionally include one or more of the following features. For example, in some embodiments, the method further includes tuning one or more system parameters based on power and performance policy settings determined for a system-on-chip processor block. The method further includes generating a data structure based on machine learning computations performed by a reinforcement learning model of the power manager. The data structure maps multiple workload types to multiple power hints based on a reward function that optimizes each power and performance policy setting for each of the multiple power hints. In some embodiments, each of the multiple power hints is mapped to a corresponding set of power and performance policy settings that represent a global power policy for the corresponding workload type.

[0008] In some embodiments, the clustering algorithm is an unsupervised K-means clustering algorithm, while in other embodiments, the clustering algorithm is a supervised K-means clustering algorithm. The method further includes i) training an ML classifier model based on a K-means clustering algorithm during the training phase of the ML classifier model; ii) identifying multiple distinct patterns from multiple datasets representing different workloads processed on a system-on-chip during the training phase; and iii) generating N clusters of workloads based on the identified multiple distinct patterns during the training phase. Each of the N clusters may contain each subset of workloads within the different workloads represented by the multiple datasets.

[0009] In some embodiments, the multiple inputs include i) a system index comprising one or more of instruction counts, processor cycles, stall events, and cache miss events; ii) data generated by one or more hardware counters; and iii) performance information generated by the system on the chip. The multiple inputs may also include one or more of i) a first data value indicating one or more software events; ii) a second data value indicating one or more system indices; and iii) a third data value indicating one or more firmware statistics.

[0010] Other aspects of the subject matter described herein can be embodied in a dedicated system-on-a-chip (SoC). The dedicated SoC includes a processing device, a power manager including at least one neural network ML model, and a non-temporary machine-readable storage medium that stores instructions executable by the processing device to result in the execution of various operations. For example, multiple operations include: acquiring multiple inputs generated by a system-on-a-chip performance counter; determining a workload type by the ML classifier model of the power manager in response to having processed the multiple inputs based on a clustering algorithm applied to the multiple inputs by the ML classifier model; identifying power hints using a data structure that maps the multiple workload types to multiple power hints based on the workload type; and determining power and performance policy settings for the system-on-a-chip processor block for the power hints.

[0011] This aspect and other embodiments of other aspects include corresponding systems, devices, and computer programs encoded on a computer storage device and configured to perform actions of the method. One or more computer systems can be configured by software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform actions when in operation. One or more computer programs can be configured by having instructions that, when executed by a data processing device, cause the device to perform actions.

[0012] Details of one or more embodiments of the subject matter described herein are given in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawing]

[0013] [Figure 1]This is a block diagram of an exemplary computing system, including an exemplary ML power manager. [Figure 2] Figure 1 shows an exemplary system architecture illustrating data routing for different use cases, including the ML Power Manager. [Figure 3] Figure 1 shows an example of a workload classifier for the ML Power Manager. [Figure 4] This document provides an exemplary flow for training a workload classification model of an ML classifier model. [Figure 5] This example demonstrates a workflow for optimizing workload-specific power policies using reinforcement learning. [Figure 6] Figure 1 shows an exemplary process for ML-based workload classification and optimized power management using the ML Power Manager. [Modes for carrying out the invention]

[0014] Similar reference numbers and designations in various drawings indicate similar elements.

[0015] Figure 1 is a block diagram of an exemplary computing system 100 including a system-on-a-chip 102 ("SoC102"). The SoC102 includes a central processing unit 104 ("CPU104"), a memory controller 105, shared memory 106 ("Memory 106"), an operating system ("OS") kernel 108, and IP / circuit blocks 110. In some embodiments, the system 100 may include multiple SoCs, and the description of SoC102 applies equally to each of the multiple SoCs that may be included in the system 100.

[0016] CPU104 may be a general-purpose CPU (e.g., a single-core or multi-core CPU). CPU104 generates one or more indicators, such as an application launch indicator or a function call triggered in response to the execution or launch of an application on a user device. For example, the application may be a camera application that uses an image sensor to generate image data. In some embodiments, the application is configured to request the camera application to obtain image or video content that can be rendered or edited in a user interface generated using the application. The application may request memory and image / graphics processing resources to modify and / or render image content associated with the application. CPU104 also generates one or more applicable values, such as pixel values ​​or frame rate information including frames per second (FPS). Applicable values ​​may be descriptions of events that occur during the execution of the application, or both, which may be associated with function calls.

[0017] Memory 106 is system memory, shared memory, or both. In the example in Figure 1A, memory 106 is shown outside the circuit block 110. However, memory 106 may include a portion of memory that is i) specific to the circuit block 110, ii) outside the circuit block 110, or iii) both. Memory 106 may be a shared random access memory resource of the SoC 102, such as static random access memory (SRAM). In some embodiments, an aspect of memory 106 is configured as a shared scratchpad memory that supports parallel access of its memory resource by two or more processors of the circuit block 110. Memory 106 may also include various other types of memory, such as high-bandwidth memory (HBM).

[0018] The OS kernel 108 is a core control interface between the exemplary hardware and software layers of system 100. The OS kernel 108 is instantiated in firmware and / or software, and its corresponding program code can be stored in the main memory of the CPU 104. In some embodiments, the OS kernel 108 is the core program of the OS that enables and / or allows the OS to simultaneously manage and control different hardware and software components of system 100. For example, the OS kernel 108 may enable interactions between software and hardware components of system 100, including input / output (I / O) devices and connections, CPU 104 and cache usage, device drivers, file systems, and network sockets.

[0019] In the example in Figure 1, the OS kernel 108 includes an ML power manager 109 that runs an ML model for more efficient, granular control of the power and / or DVFS settings of the system 100, particularly the SoC 102. The ML power manager 109, including its associated control logic, runs in hardware, software, or both. For example, the ML power manager 109 may include resources such as flip-flops, registers, and buffers that run in hardware, and control logic (e.g., programmed code) that runs in software. Embodiments of the ML power manager 109 can also run as firmware for the SoC 102 or as firmware for devices of the SoC 102 (such as the CPU 104, a dedicated processor for the IP / circuit block 110, or both).

[0020] A circuit block 110 generally includes individual IP devices such as processors, processor cores, or dedicated processing devices. For example, a circuit block 110 may include an image signal processor (ISP) 112, a dedicated (SP) processor 114, a digital signal processor (DSP) 116, and a graphics processing unit (GPU) 118. The SP processor 114 may include one or more processing units / devices and is instead referred to as the SP processing unit 114 ("SPU 114"). A circuit block 110 is instead referred to as an IP block 110, and an IP block may include one or more proprietary hardware elements. For example, each of the ISP 112, SP processor 114, DSP 116, and GPU 118 may be proprietary IP blocks (or IP devices) of a particular entity or device manufacturer.

[0021] The SP processor 114 is an ML hardware accelerator, neural network (NN) processor, or related processor designed and optimized to implement and / or run ML models for computing inference on edge devices. In some embodiments, the SP processor 114 (or SPU 114) is a tensor processing unit ("TPU") that includes a hardware architecture optimized for traversing tensors or other multidimensional constructs to process inputs through neural network layers using corresponding parameter sets or kernel filters for the weights of neural network layers. An exemplary hardware architecture for such an SP processor 114 (or TPU) is described in U.S. Patent No. 10,175,980 B2, which is incorporated herein by reference in its entirety.

[0022] In some embodiments, aspects of the ML power manager 109 are implemented as software routines (or modules) of the SP processor 114. In some examples, these aspects can also be executed as software routines of the CPU 104 using one or more hardware resources of the CPU 104 such as registers, buffers, etc. The SP processor 114 can be configured as an instruction and vector data processing engine that processes data obtained from the system memory of the SoC 102 such as the memory 106. In some embodiments, the ISP 112 and the SP processor 114 are the same device, integrated in a single device package, or both. In other embodiments, the ISP 112 and the SP processor 114 are separate devices formed on separate dies within the same integrated circuit package or different integrated circuit packages.

[0023] Each processor of the SoC 102 (e.g., ISP 112, DSP 116, SPU 114, GPU 118) can include multiple cores. The OS kernel 108, the memory controller 105, and / or the CPU 104 can generate control signaling to manage memory-intensive computing operations and distribute them to the memory device 122 (e.g., DRAM), for example, to minimize the processing load at each core of the processor. The OS kernel 108, the memory controller 105, and / or the CPU 104 can generate control signaling to manage and distribute operations according to a global power policy, for example, to achieve specific performance requirements, power consumption thresholds, or both. The control signaling is routed in the system 100 using an exemplary bus 120 of the SoC 102. The control signals can include commands, requests, data, instructions, or combinations thereof.

[0024] The OS kernel 108 and the ML power manager 109 cooperate with at least the SP processor 114 and, optionally, other devices such as the CPU 104 and the IP block 110 to perform real-time ML-based workload classification and power management optimization. More specifically, the ML power manager 109 is configured to generate control signaling to cooperate with the OS kernel 108 to configure, manage, and execute the global power policy of the system 100.

[0025] In some embodiments, the operations are executed on the SoC 102 to support heterogeneous computing operations between two or more processing units included in the IP block 110, the CPU 104, or both. For example, the heterogeneous computing operations can occur between the ISP 112 and the SP processor 114. The ML power manager 109 performs its power management operations using an ML model that includes one or more neural network ML models.

[0026] For example, the ML power manager 109 is configured to execute an ML workload classifier model and a power optimization ML model. Each of the ML workload classifier model and the power optimization ML model can be a neural network ML model executed on the SoC 102 using the SP processor 114. The ML workload classifier model is described below with reference to FIGS. 3 and 4, while the power optimization ML model is described below with reference to FIG. 5.

[0027] In some embodiments, the system 100 includes an exemplary memory device 122. The memory device 122 can be a dynamic random access memory (DRAM) or a double data rate (DDR) synchronous DRAM (SDRAM). The memory device 122 is configured to perform or support various types of processor-in-memory ("PiM") operations, compute-in-memory ("CiM") operations, and memory near computing operations ("MnC") operations. The memory device 122 uses its plurality of PiM computing blocks to perform or support these operations.

[0028] PiM, CiM, and MnC operations include standard arithmetic operations such as calculations typically performed by the ALU or MAC. For example, these operations may also include the computational capabilities of the SPU114, such as matrix mathematics, vector calculations, linear algebra, and multiplication and addition operations for dot product accumulation. In some embodiments, each of the PiM, CiM, and MnC operations is performed to support machine learning calculations, neural network calculations, or both. Exemplary PiM, CiM, and MnC operations may also include standard CPU functions involving instruction processing, interrupt handling, and branch and pointer operation management.

[0029] The SoC102 works with the memory device 122 to perform data storage and PiM / CiM computations across one or more memory dies of the memory device 122. As described above, the computations may be for heterogeneous operation or machine learning workloads involving one or more processors in the IP block 110. In at least one example, the memory device 122 may be external to the SoC102, while in other examples, the memory device 122 may be internal to the SoC102.

[0030] System 100 includes a multi-camera system 124. The multi-camera system 124 includes at least a first camera 126 having a wide-angle (or ultra-wide-angle) lens for capturing image content within a first FOV, and a second different camera 128 having a telephoto lens for capturing image content within a second different FOV. The second FOV of the second camera 128 is narrower than the first FOV of the first camera 126. System 100 is configured to merge images captured by the different cameras of the multi-camera system 124. In some embodiments, the first camera 126 may be described as a wide-angle camera (W), while the second camera 128 may be described as a telephoto camera (T).

[0031] In the example shown in Figure 1, the multi-camera system 124 is integrated into an exemplary edge device such as a client / mobile device 130 (described below). In particular, the multi-camera system 124 could be an exemplary camera system in a modern mobile smartphone, which is a well-known hardware feature of edge devices. In some embodiments, the multi-camera system 124 includes a time-of-flight (ToF) sensor to enhance the image capture performance of the multi-camera system 124. For example, the image capture performance is enhanced under ambient light conditions, but power consumption is reduced compared to a multi-camera system that does not include a ToF sensor.

[0032] In the example in Figure 1, System 100 and SoC 102 are integrated circuits for exemplary user / client devices 130, consumer electronics devices, or mobile devices, while each of these devices may include items such as smartphones 130a, tablets 130b, laptops 130c, smartwatches, and wearable devices. Device 130 may also include other items such as e-notebooks, netbooks, and / or related mobile computing devices (including unmanned aerial vehicles and other autonomous vehicles, automobiles, self-driving cars, and systems for controlling autonomous vehicles).

[0033] Figure 2 shows an exemplary system architecture 200 illustrating data routing for different use cases, including the ML Power Manager from Figure 1.

[0034] The system architecture 200 can be used for different use cases 202 that may be executed or encountered on the client / mobile device 130. Based on the different use cases 202, the ML power manager 109 can receive data corresponding to specific events for each of the use cases 202. For example, use case 202 could be real-time navigation on the mobile device 130. In this example, events may include opening a navigation app, determining the device's location, rendering a map on the screen, etc. In some embodiments, the events are global SoC events generated when executing the application program for a given use case 202.

[0035] The ML Power Manager 109 can be configured as a module of the OS kernel 108, designed to achieve the shortest latency and minimal resource overhead when determining the power and performance policy settings of the SoC and acquiring the necessary inputs to manage other power subsystems. The ML Power Manager 109 may include a machine learning-based workload classifier as the governor of the power management processes running on the SoC 102. The workload classifier is described in more detail below with reference to Figures 3 and 4.

[0036] In addition to receiving data representing use cases 202 and events associated with each use case 202, the ML Power Manager 109 can also receive activity statistics related to use cases 202. For example, the ML Power Manager 109 can receive activity statistics from other components of the system 100 that communicate with the OS kernel 108, as well as from the chip power model (CPM) and firmware of the SoC 102.

[0037] In some embodiments, CPM is a model used to estimate and manage the power consumption of the SoC102 chip or IP block. Activity statistics may include software events, system indices, low-level firmware statistics, and workloads. As a specific example, software events may include events from system servers, frameworks, or applications. As another example, system indices may include CPU / GPU / MIF frequencies or active ratios. As a specific example, low-level firmware statistics may include metrics from the Performance Monitoring Unit (PMU), Platform Performance Measurement Unit (PPMU), and / or GPU hardware counters.

[0038] Next, the ML Power Manager 109 can use the activity statistics for use case 202 as input data to be processed in order to classify the workload of a given use case and to enhance or optimize power management decisions in the SoC 102. By receiving activity statistics from both the OS kernel 108 and the firmware of the SoC 102, the ML Power Manager 109 determines power and performance policy settings and DVFS control values ​​based on a global view of the system 100, and further optimizes power management within the system 100. Workload classification using the ML Power Manager is described in more detail below with reference to Figures 3 and 4.

[0039] The ML Power Manager 109 is configured to generate power hints based on use case 202 and system activity statistics when executing a given use case. In some embodiments, power hints are provided as input to one or more kernel components, such as the CPU Scheduler and Dynamic Voltage and Frequency Scaling (DVFS), GPU DVFS, Memory DVFS, ISP DVFS, and DPU DVFS. In some embodiments, power hints represent the setting of predefined power and performance policies for specific events, such as camera shot events or game events.

[0040] ML Power Manager 109 can generate these power hints to improve power management within the system by influencing processor power and performance policy settings for specific events. For example, it can generate power hints for graphics processing units (GPUs) that provide predefined optimized power settings to best balance power efficiency and performance output for rendering maps on screen during navigation use cases.

[0041] For example, power hints may indicate threshold voltages and frequencies to each DVFS module that improve overall power management and power allocation in the SoC102 while simultaneously optimizing performance. In this particular example, power hints could influence the GPU DVFS to increase power to the GPU up to a certain level in order to achieve a threshold rendering speed or frame rate for rendering the map graphics. Using the DVFS settings, the processor voltage and frequency can be dynamically adjusted, and each DVFS can use these power hints to optimally adjust its frequency to match the power efficiency and performance optimized for a particular processor. In some embodiments, each processor in the SoC can perform its own DVFS control to enable finer-grained control over power consumption in that processor in a way that improves overall power management in the system 100.

[0042] Figure 3 shows an exemplary workload classifier of the ML Power Manager described above, with reference to Figure 2. The workload classifier 300 is designed to dynamically optimize the power policy based on real-time system metrics collected by various hardware counters. The workload classifier 300 may include a statistics collector 304, an ML classifier 302, and a hint manager 306. The workload classifier 300 can run in hardware and software. For example, some embodiments of the workload classifier 300 can run in software, while other embodiments can run in hardware.

[0043] The statistics collector 304 is configured to collect real-time system metrics. For example, the statistics collector 304 can collect low-level system indices from the Performance Monitoring Unit (PMU) 310, including instruction counts, CPU cycles, IPC, stalls, and L3 cache misses. In some embodiments, the statistics collector 304 can be integrated with a Platform Performance Measurement Unit (PPMU) 312 and other hardware counters (e.g., GPU hardware counters 314) to enhance data collection with additional performance metrics. That is, the statistics collector 304 can collect additional metrics that are analyzed by the PPMU and other hardware counters.

[0044] Next, the ML classifier 302 can classify the workload based on the metrics collected from the statistics collector 304. The ML classifier 302 can classify the workload using a decision tree. The decision tree can be trained offline, which is explained in more detail with reference to Figure 4. Then, the ML classifier 302 can determine an appropriate power management scheme based on the classified workload.

[0045] The Hint Manager 306 is configured to generate and / or maintain a workload-hint mapping table 308. The Hint Manager 306 uses the workload-hint mapping table 308 to map classified workloads to specific power management hints known as PowerHints. PowerHints can be software hints that predefine power and performance policy settings for specific use cases (e.g., camera shots, games). Each PowerHint may have a corresponding set of power and performance policy settings and / or performance settings that are triggered on the SoC based on a specific workload type. PowerHints can then be used to pass power and performance policy settings (e.g., DVFS settings) to the kernel to influence the CPU scheduler and other governors (e.g., DVFS modules).

[0046] In some embodiments, PowerHint may be represented as an uclamp_min value in the workload-hint mapping table 308. The uclamp_min value can set a lower limit for the performance requirements of a particular task. That is, uclamp_min can tell the CPU scheduler the minimum performance point at which it should operate to provide the desired user experience for a particular task. In this embodiment, the table maps specific values ​​to the uclamp_min value in order to optimize power management.

[0047] In some embodiments, the workload-hint mapping table 308 may support additional tunable values ​​for a more comprehensive optimization scheme. Hereinafter, the power / performance / DVFS setting may also be referred to as the global power policy corresponding to a specific PowerHint. That is, the workload type to the PowerHint mapping table can determine the power policy to run within the SoC.

[0048] The workload-hint mapping table is a data structure that maps specific workload types to specific PowerHints. The hint manager 306 can apply these hints to tune system parameters, including CPU frequency, thread scheduling, and power states. The workload-hint mapping table 308 can be trained offline to optimize the mapping of PowerHints to workload types. The workload-hint mapping table is described in more detail with reference to Figure 5.

[0049] Next, the workload classifier 300 can select a specific PowerHint based on the workload type by reading the optimized workload-PowerHint mapping table. Then, the workload classifier can adjust power and performance policy settings (e.g., DVFS settings) based on the given PowerHint to implement a specific global power policy. Essentially, the workload classifier 300 can provide more adaptive real-time classification and power management by dynamically adjusting power and performance policy settings based on the classified workload and optimized PowerHint.

[0050] Figure 4 shows an exemplary flow for training a workload classification model of an ML classifier model. More specifically, workload classification model training is a method for training a machine learning model for workload classification.

[0051] First, one or more workload collections 402 are collected. A workload collection, for example, workload collection 404, is a dataset containing system indexes, software events, firmware state, and various PMU counter metrics (e.g., instructions, cycles, IPC, stalls, cache misses, and other related metrics). Metrics can be captured under different operating conditions and use cases, and each of the one or more workload collections 402 represents a different type of system workload. For example, workload collection 404 may represent a web browsing workload. Other workload collections may represent any appropriate workload for the user / client device's SoC, including but not limited to streaming video, games, voice calls, navigation, or music playback.

[0052] Next, the model training framework 408 can receive one or more workload collections 402. The model training framework 408 can train a workload classifier model 410 to classify one or more workloads 402 into workload classification types 412. The model training framework 408 can train the workload classifier 410 using machine learning techniques (e.g., unsupervised or supervised learning).

[0053] In some embodiments, the model training framework 408 can train a workload classifier 410 using unsupervised learning. For example, the model training framework 408 can use k-means clustering to identify patterns in one or more workloads 402 and group the workloads into N clusters or M workload types. K-means clustering works by randomly selecting K initial centroids and then assigning each data point to the nearest centroid to form clusters. In some embodiments, the “nearest” centroid can be calculated based on Euclidean distance. The centroids are recalculated over the assignment period as the average of the data points in each cluster.

[0054] In this specific example, each cluster represents a distinct type of workload characterized by specific system behavior and resource requirements. That is, one or more workload collections 402 are assigned to a particular workload cluster based on similar characteristics of the workload collections. For example, workload collections 402 can be clustered based on instructions per cycle (IPC), IPC time variance, memory frequency, and GPU workloads. By using k-means clustering, the model training framework 408 can group workloads with similar characteristics together into N clusters or M workload classification types 412. By defining classification types 412 based on characteristics rather than more general workload types (e.g., games), workload collections 402 are analyzed and classified at a finer granularity, enabling more effective optimization and improved power management. For example, not all game workloads are the same and should not be classified in the same way. The approach described herein takes this fact into account and further focuses on system behavior and resource requirements as described above, resulting in a more accurate and optimized classification for subsequent power policy management.

[0055] In some embodiments, the workload classifier 410 can be trained using supervised learning. For example, the workload classifier 410 can be trained using logistic regression or the k-nearest neighbor (KNN) algorithm. The workload classifier 410 is not limited to these methods and can be trained using any suitable supervised learning method. The model training framework 408 can then output a machine learning model (e.g., the workload classifier 410) that encapsulates the classification type 412 and serves as a foundation for classifying workloads in real time.

[0056] The workload classifier 410 can be converted from the workload classification type 412 into a decision tree structure. A decision tree is a machine learning model configured to classify data using decisions based on data features. Each internal node represents a feature-based decision, and each leaf node represents the final classification type. Essentially, data moves through the tree based on decisions, for example, moving to the left branch of the tree if it has the feature, or to the right branch of the tree if it does not have the feature, and finally ending up at a leaf node that has the final classification based on the feature decision. For example, as described above, instructions per cycle can be a feature that influences the classification of the workload. In this specific example, the decision at the node could be whether the workload has an IPC greater than or less than 400, for example, moving to the left branch if the IPC is greater than 400, or moving to the right branch if the IPC is less than 400.

[0057] For example, in the k-means clustering example, the workload collection 402 is divided into N clusters or M workload classification types 412. As described above, each workload set 404 is assigned to a cluster based on similar features. Next, a decision tree can be trained on the classification types 412, the features associated with those classification types, and the workloads 402 clustered by classification type. The decision tree can then learn to predict the classification type 414 of the input workload collection 404 based on the features. Finally, the resulting decision tree can be used to classify a new workload collection 402 into a predefined workload classification type 412.

[0058] Decision trees can be coded in C to ensure seamless integration with the system's kernel module, enabling real-time workload classification during system operation.

[0059] Next, the workload collection 402 can be classified into a workload classification type 412 using the decision tree within the workload classifier 410. The workload classification type 412 can range from 1 to N different types. As described above, the workload classification type 412 is based on similar characteristics of the workload collection 402. For example, the workload collection 404 is collected by the model training framework 408 and can be classified as type I414 based on the workload classifier 410.

[0060] In k-means clustering, workload classification type 412 is clustered based on various features of workload collection 402, and type 412 may not have a clear meaning. For example, type 414 may not necessarily be a workload with an IPC of 300. Workload classification type 412 is clustered by similar features, but the clustering may not have an explicit meaning that is understandable to humans, as well as it may be understandable to machine learning models.

[0061] Figure 5 shows an exemplary flow of workload-specific power policy optimization using reinforcement learning.

[0062] Workload-specific power policy optimization using reinforcement learning is a method that uses reinforcement learning to determine the potentially best mapping of PowerHints to workload classification types (e.g., workload classification type 504). A reinforcement learning framework 502 can be used to generate workload types into a PowerHint mapping table. For example, PowerHints may be represented as uclamp_min values, and power management can be optimized using the workload type-uclamp_min mapping table.

[0063] Process 500 begins with a minimum test set 506 representing a set of benchmark use cases, each representing a real-world workload. The minimum test set 506 can be run by an automated test infrastructure that evaluates the system's performance under different PowerHint mapping configurations. A reinforcement learning framework 502 can generate various mapping tables in each mapping table, where potential workload types are mapped to PowerHints. The mapping tables are used to propose different PowerHint mapping configurations to implement different power management approaches across the SoC. A test cycle may include testing one or more use cases against different mapping tables to evaluate the performance within System 100 against different hints and corresponding power settings in the mapping tables. In some embodiments, a single test cycle includes testing the use cases in Test Set 506 against the current mapping table and, furthermore, against various other potential mapping tables, to evaluate the performance within System 100.

[0064] Each use case in test set 506 is executed to collect data on system behavior, including power consumption and performance metrics of the PowerHint mapping configuration. That is, the current mapping table is compared to different potential mapping tables created by framework 502 based on their effectiveness for power consumption, latency, junk, and other relevant metrics for a given use case. This cycle is then repeated for each use case in test set 506.

[0065] In some embodiments, to reduce the complexity of testing, only one workload type is boosted in each mapping table. For example, with one hint, uclamp_min might be boosted to 500, and by corresponding to 10 workload types, there could be a total of 10 mapping tables. Process 500 then uses the collected performance and power metrics and a reinforcement learning framework 502 to determine the optimal power policy (e.g., PowerHint) for each workload type 504 based on the use case. For example, use case A is tested to collect performance-power metrics for each of the potential mapping tables created.

[0066] The reinforcement learning framework 502 can receive metrics and analyze the data to evaluate the current PowerHint settings. The reinforcement learning framework 502 can then adjust the PowerHint mapping to the PowerHint mapping table 510 for the current workload type based on performance feedback from other potential mapping tables to improve both the performance and power efficiency of the power policy.

[0067] The reinforcement learning framework 502 can use reinforcement learning techniques to tune PowerHints, generate workload types into a potentially optimal PowerHint mapping table, and learn from each test cycle to enhance system optimization. For example, the reinforcement learning framework 502 can perform mappings based on a specific reward function. The reward function can be associated with the minimum power and / or frequency required to achieve a specific expected performance for a given workload classification type 504. For example, the reward function can be associated with specific values ​​of frames per second (FPS), power consumption, junk, and latency for each workload type. As a specific example, the reward function can be associated with finding the optimal PowerHint for a particular FPS value, which means finding a PowerHint with the minimum power and / or frequency required to achieve the expected frames per second (FPS).

[0068] To optimize a given reward function, for example, FPS, the reinforcement learning framework can receive performance and power hints based on the current power hint settings and update the power hint settings based on the current value of the reward function. The reinforcement learning framework 502 can then learn from the updates to determine which settings yield the best power optimization for a particular workload classification type 504, as it creates various potential mapping tables to test in the next test cycle. The framework 502 balances exploration (e.g., trying new power hint settings) and exploitation (e.g., using known good settings). For example, the reinforcement learning framework 502 can adjust a power hint to map to workload type I and verify that it does not optimize power. The reinforcement learning framework 502 can then adjust workload type I to map to other hints, and further learn that the power optimization has worsened as a result of that adjustment, so that it can be avoided in the future. As another example, the reinforcement learning framework 502 can map a hint to workload type I and verify that the performance / power metrics have improved, and continue to adjust the mapping for further optimization.

[0069] By optimizing workload-specific power policies using reinforcement learning, PowerHint can be iteratively improved with each test cycle to optimize system performance and power efficiency.

[0070] After a cycle has occurred, an optimized global power policy (e.g., power settings and performance settings) is determined for each workload type based on the optimized hints. More specifically, the reinforcement learning framework 502 generates potentially optimal workload types 504 in the PowerHint mapping table. For example, a workload classified as type I is mapped to a specific PowerHint (e.g., PowerHint II). For each workload classified as type I, the PowerHint is the same, which in the given example is PowerHint II.

[0071] As described above with reference to Figure 3, each PowerHint has a corresponding set of power / performance / DVFS settings that can be triggered on the SoC based on a specific workload type. Therefore, a global power policy can be determined based on the workload type by mapping the workload type to an optimized PowerHint corresponding to a specific power and performance policy setting within the SoC.

[0072] To evaluate other mappings in subsequent test cycles, the reinforcement learning framework 502 can generate more potential workload-PowerHint mapping tables based on the pre-optimized mapping table from previous training iterations. The various workload-PowerHint mapping tables are then reapplied to the next use case in a minimum test set 506. Essentially, a continuous feedback loop is established in which the reinforcement learning framework incrementally improves PowerHints based on actual word data, optimizing both power efficiency and performance.

[0073] Figure 6 shows an exemplary process 600 for ML-based workload classification and power management optimization using the ML Power Manager 109. Process 600 is also implemented or executed on system 100 using the OS kernel 108 and / or the ML Power Manager 109 as described above with reference to at least Figure 1. Thus, the description of process 600 refers to the computing resources of system 100 as described above. In some examples, the steps or actions of process 600 are enabled by programmed software instructions, firmware instructions, or both. Each type of instruction may be stored in a non-temporary machine-readable storage device and can be executed by one or more of the processors or other resources described herein.

[0074] Referencing process 600 again, system 100 receives inputs generated by system-on-chip performance counters coupled to the power manager (602). System 100 determines the workload type in response to processing the inputs, based on a clustering algorithm applied to the inputs by the ML classifier model of the ML power manager 109 (604). System 100 identifies power hints based on the workload type (606). For example, the ML power manager 109 can identify power hints using a data structure that maps different workload types to different power hints.

[0075] For a particular workload type, system 100 is configured to determine power and performance policy settings on the system-on-chip based on identified power hints mapped to that particular workload type (608). For example, the ML Power Manager 109 can determine power and performance policy settings for a particular processor block or multiple IP blocks on the system-on-chip based on identified power hints. System 100 adjusts one or more system parameters based on the power and performance policy settings determined for the processor block on the system-on-chip (610). For example, system 100 uses the ML Power Manager 109 to perform any necessary adjustments to a given power setting in order to implement a particular global power policy corresponding to an identified power hint.

[0076] In some embodiments, each step of process 600 is performed system-on-chip as part of a large-scale computation operation to generate a machine learning (ML) output, which includes the output of the neural network layer of a neural network running one or more machine learning (ML) models. For example, the output may be part of a computation for an ML task or inference workload to generate an image processing, speech processing, or image recognition output. As shown above, part of the integrated circuit may be a dedicated inference processor (such as a neural network processor or hardware ML accelerator) configured to accelerate computations to generate different types of data processing outputs.

[0077] The subject matter and functional embodiments described herein can be executed in digital electronic circuits, tangibly embodied computer software or firmware, computer hardware, or in one or more combinations thereof, including structures and structural equivalents thereof disclosed herein. The embodiments of the subject matter described herein can be executed by a data processing device or as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a tangible non-transient program carrier to control the operation of a data processing device.

[0078] Alternatively or additionally, program instructions can be encoded into artificially generated propagating signals (e.g., mechanically generated electrical, optical, or electromagnetic signals) that are produced to encode information for transmission to a receiving device suitable for execution by a data processing device. Computer storage media may be machine-readable storage devices, machine-readable storage boards, random or serial access memory devices, or a combination of one or more of these.

[0079] The term “computing system” includes, for example, all kinds of devices, machines, and apparatus for processing data, including programmable processors, computers, or multiple processors or computers. Apparatus may include dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, apparatus may also include code that constitutes the execution environment for the computer program in question, such as processor firmware, protocol stacks, database management systems, operating systems, or one or more of these.

[0080] Computer programs (which may be called, or described as, programs, software, software applications, modules, software modules, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, such as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0081] Computer programs can, but not always, correspond to files in a file system. A program can be stored in a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program itself, or in multiple collaborative files (e.g., files storing one or more modules, subprograms, or parts of code). Computer programs can be deployed to run on one computer or on multiple computers, which may be located in a single site or distributed across multiple sites and interconnected by a communication network.

[0082] The processes and logic flows described herein can be performed by one or more programmable computers running one or more computer programs to perform their functions by manipulating input data and generating outputs. The processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), or GPGPUs (General-Purpose Graphics Processing Units), and the devices can also be performed as such.

[0083] A computer suitable for running computer programs may include, for example, a general-purpose or dedicated microprocessor, or both, or any other type of central processing unit, and may be based on them. Generally, a central processing unit receives instructions and data from read-only memory, random-access memory, or both. Components of a computer include a central processing unit for issuing and executing instructions, and one or more memory devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or is operablely connected to receive data from or transmit data to such mass storage devices, or both. However, a computer is not required to have such devices. Furthermore, a computer can be incorporated into other devices, to name a few, such as a mobile phone, a personal digital assistant (PDA), a portable audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a Universal Serial Bus (USB) flash drive).

[0084] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, which include, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory can be complemented by or incorporated into dedicated logic circuits.

[0085] To provide user interaction, embodiments of the subject matter described herein can be performed on a computer having a display device for displaying information to the user (e.g., an LCD monitor) and a keyboard and pointing device (e.g., a mouse or trackball) on which the user can provide input to the computer. Interaction with the user can similarly be provided using other types of devices. For example, the feedback provided to the user may be any form of sensory feedback (e.g., visual, auditory, or tactile feedback), and input from the user may be received in any form, including acoustic, speech, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from devices used by the user, for example, by sending a web page to a web browser on the user's client device in response to a request received from a web browser.

[0086] Embodiments of the subject matter described herein can run within a computing system that includes a backend component (e.g., as a data server), or a middleware component (e.g., an application server), or a frontend component (e.g., a client computer having a graphical user interface or a web browser on which a user can interact with embodiments of the subject matter described herein), or in any combination of one or more such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communications (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs") (e.g., the Internet).

[0087] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact via a communication network. The relationship between clients and servers arises from computer programs running on each computer that have a client-server relationship with one another.

[0088] While this specification includes details of many specific embodiments, these should not be interpreted as limitations on the scope of any invention or claim, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Furthermore, certain features described herein in the context of separate embodiments can be performed in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment can be performed separately or in any suitable subcombination in multiple embodiments. Moreover, even if features are described above as functioning in a particular combination, and may initially be claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may be subject to subcombinations or variations of subcombinations.

[0089] Similarly, while operations are shown in a specific order in the drawings, this should not be understood as meaning that such operations must be performed in a specific order or sequence shown, or that all shown operations must be performed, in order to achieve the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as meaning that such separation is required in all embodiments, and the described program components and systems should generally be understood as being able to be integrated into a single software product or packaged into multiple software products.

[0090] Specific embodiments of the subject matter are described. Other embodiments are within the scope of the claims below. For example, the actions described in the claims may be performed in a different order to achieve more desirable results. As an example, the process shown in the accompanying figures does not necessarily require to be in a specific or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A method performed using a power manager that includes one or more neural network machine learning (ML) models, Acquire multiple inputs generated by a system-on-chip performance counter coupled to the power manager, Based on a clustering algorithm applied to the multiple inputs by the ML classifier model, the ML classifier model of the power manager determines the workload type in response to processing the multiple inputs. Based on the aforementioned workload type, a data structure is used to identify power hints by mapping multiple workload types to multiple power hints, Regarding the aforementioned power hint, it determines the power and performance policy settings for the processor block of the system-on-chip, Methods that include...

2. The method according to claim 1, further comprising adjusting one or more system parameters based on the power and performance policy settings determined for the processor block of the system-on-chip.

3. The process further includes generating the data structure based on machine learning computations performed by the reinforcement learning model of the power manager, The method according to claim 2, wherein the data structure maps the plurality of workload types to the plurality of power hints based on a reward function that optimizes each of the plurality of power hints' power and performance policy settings.

4. The method according to claim 3, wherein each of the aforementioned power hints is mapped to a corresponding set of power and performance policy settings that represent a global power policy for the corresponding workload type.

5. The method according to claim 1, wherein the clustering algorithm is an unsupervised K-means clustering algorithm.

6. The method according to claim 1, wherein the clustering algorithm is a supervised K-means clustering algorithm.

7. During the training phase of the aforementioned ML classifier model, the ML classifier model is trained based on the K-means clustering algorithm. During the aforementioned training phase, the system identifies multiple distinct patterns from multiple datasets representing different workloads processed on the system-on-chip, During the training phase, N clusters of workloads are generated based on the identified multiple distinct patterns. The method according to claim 6, further comprising:

8. The method according to claim 7, wherein each of the N clusters comprises a subset of the workloads within the different workloads represented by the plurality of datasets.

9. The aforementioned multiple inputs are, i) A system index including one or more of the instruction count, processor cycles, stall events, and cache miss events, ii) Data generated by one or more hardware counters, iii) Performance information generated by the system-on-chip, The method according to claim 1, including the method described in claim 1.

10. The aforementioned multiple inputs are, i) A first data value representing one or more software events, ii) A second data value that indicates one or more system indexes, iii) A third data value representing one or more firmware statistics, The method according to claim 1, comprising one or more of the above.

11. It is a dedicated system-on-a-chip, Processing device and A power manager containing at least one neural network ML model, The system comprises a non-temporary machine-readable storage medium that stores instructions executable by the processing device in order to cause an operation to be performed, and the operation is, The acquisition of multiple inputs generated by the system-on-chip performance counter, Based on a clustering algorithm applied to the multiple inputs by the ML classifier model, the ML classifier model of the power manager determines the workload type in response to processing the multiple inputs. Based on the aforementioned workload type, a data structure is used to identify power hints by mapping multiple workload types to multiple power hints, Regarding the aforementioned power hint, it determines the power and performance policy settings for the processor block of the system-on-chip, Includes a dedicated system-on-a-chip.

12. The aforementioned operation is, The dedicated system-on-chip according to claim 11, further comprising adjusting one or more system parameters based on the power and performance policy settings determined for the processor block of the system-on-chip.

13. The aforementioned operation is, The process further includes generating the data structure based on machine learning computations performed by the reinforcement learning model of the power manager, The dedicated system-on-chip according to claim 12, wherein the data structure maps the plurality of workload types to the plurality of power hints based on a reward function that optimizes the respective power and performance policy settings for each of the plurality of power hints.

14. Each of the aforementioned power hints is mapped to a corresponding set of power and performance policy settings representing a global power policy for the corresponding workload type, as described in claim 13.

15. The dedicated system-on-chip according to claim 11, wherein the clustering algorithm is an unsupervised K-means clustering algorithm.

16. The dedicated system-on-chip according to claim 11, wherein the clustering algorithm is a supervised K-means clustering algorithm.

17. The aforementioned operation is, During the training phase of the aforementioned ML classifier model, the ML classifier model is trained based on the K-means clustering algorithm. During the aforementioned training phase, the system identifies multiple distinct patterns from multiple datasets representing different workloads processed on the system-on-chip, During the training phase, N clusters of workloads are generated based on the identified multiple distinct patterns. The dedicated system-on-chip according to claim 16, further comprising:

18. The dedicated system-on-chip according to claim 17, wherein each of the N clusters includes a subset of the workloads within the different workloads represented by the plurality of datasets.

19. The aforementioned multiple inputs are, i) A system index including one or more of the instruction count, processor cycles, stall events, and cache miss events, ii) Data generated by one or more hardware counters, iii) Performance information generated by the system-on-chip, A dedicated system-on-chip according to claim 11, including the above.

20. A non-temporary machine-readable storage device for storing instructions for executing a power manager that runs on a dedicated system-on-chip, wherein the power manager includes at least one neural network ML model, the instructions are executable by a processing device to cause an operation to be performed, and the operation is The acquisition of multiple inputs generated by the system-on-chip performance counter, Based on a clustering algorithm applied to the multiple inputs by the ML classifier model, the ML classifier model of the power manager determines the workload type in response to processing the multiple inputs. Based on the aforementioned workload type, a data structure is used to identify power hints by mapping multiple workload types to multiple power hints, For the workload type, the power and performance policy settings for the system-on-chip processor block are determined based on the identified power hints. Non-temporary machine-readable storage devices, including [specific type of storage device].

Images