Machine learning based power consumption and performance optimization system and method for graphics processing unit

Abstract

Systems and methods for dynamically adjusting the operating state of a graphics processing unit (GPU) are disclosed. A machine learning model is trained to determine an operating voltage and frequency to be provided to a GPU core of the GPU to perform a work task including a plurality of instructions. And deploying the trained machine learning model to firmware of the GPU. Instructions in a work task to be executed by a GPU core are received. And the trained machine learning model determines the working voltage and frequency of the GPU core execution instruction.

Description

Technical Field

[0001] The present invention relates to processor power management. In particular, the present invention relates to optimizing the power consumption delivered to a graphics processing unit (GPU). Background Art

[0002] Current implementations of power optimization in integrated circuits are usually based on dynamic voltage and frequency scaling (DVFS). In DVFS, a processor such as a central processing unit (CPU) or a graphics processing unit (GPU) adjusts its operating voltage and frequency based on past workloads to meet the needs of current or subsequent workloads. This type of DVFS scheme is usually not suitable for GPU workloads that change instantaneously, because the adjusted voltage and frequency will not match the needs of each workload and may result in energy consumption and performance loss. Summary of the invention

[0003] This article describes a system and method for dynamically adjusting the working state of a GPU. A machine learning model can be trained to determine the operating voltage and frequency of a GPU core to be provided to a GPU to perform a work task containing multiple instructions. The trained machine learning model can be deployed to the firmware of the GPU. Instructions to be executed by the GPU core in the work task can be received. The trained machine learning model can determine the operating voltage and frequency of the GPU core to execute the instructions.

[0004] In some embodiments, before the GPU core executes the instruction, the determined operating voltage and frequency may be provided to the GPU core.

[0005] In some embodiments, the machine learning model may include a voltage-frequency inference model that is trained to determine the operating voltage and frequency of the GPU core when executing instructions.

[0006] In some embodiments, the machine learning model may further include a rendering performance inference model that is trained to determine performance data of the GPU core when executing instructions.

[0007] In some embodiments, the machine learning model also includes a power consumption inference model that is trained to determine the power consumption of the GPU core when executing instructions.

[0008] In some embodiments, if the determined power consumption exceeds a threshold, the operating voltage and frequency of the GPU core may be adjusted.

[0009] In some embodiments, performance data of the GPU core when executing the instruction can be determined based on the instruction. Power consumption of the GPU core when executing the instruction can be determined based on the instruction. An optimal operating voltage and frequency for the GPU core to execute the instruction can be determined based on the instruction, the determined performance data, and the determined power consumption.

[0010] In some embodiments, a first data set generated by a GPU simulation model may be used to train a machine learning model. The first data set may include data associated with simulation instructions of a work task, simulation operating voltage, simulation operating frequency, simulation power consumption, and simulation performance data of the GPU core when executing the simulation instructions.

[0011] In some embodiments, a second data set generated by a GPU simulation model may be used to train a machine learning model. The second data set may include data associated with simulation instructions of the work task, simulation operating voltage, simulation operating frequency, simulation power consumption, and simulation performance data of the GPU core when executing the simulation instructions.

[0012] In some embodiments, a second instruction to be executed by the GPU core may be received. The trained machine learning model may determine a second operating voltage and frequency for the GPU core to execute the second instruction based on the second instruction.

[0013] In some embodiments, the instructions of the work task may include at least drawing instructions for rendering three-dimensional graphics.

[0014] In some embodiments, the trained machine learning model is not based on a feedback mechanism in which the trained machine learning model provides an operating frequency and voltage to the GPU core to perform a current work task based on the operating frequency and voltage of the GPU core when performing a previous work task.

[0015] These and other features of the apparatus, system, method and non-transitory computer readable medium disclosed herein, as well as the methods of operation and functions of the related elements of structure, combination of parts and manufacturing economy will become more apparent after considering the following description and the appended claims with reference to the accompanying drawings. All drawings constitute a part of this specification and the same reference numerals in the various figures indicate corresponding parts. However, it should be expressly understood that the drawings are for illustration and description purposes only and are not intended to define the limitations of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Certain features of various embodiments of the present technology are set forth with particularity in the appended claims.The features and advantages of the present technology may be better understood by reference to the following detailed description, which sets forth illustrative embodiments utilizing the principles of the invention, and the accompanying drawings.

[0017] Figure 1 A conventional dynamic voltage-frequency adjustment scheme is shown.

[0018] Figure 2 An improved dynamic voltage-frequency adjustment scheme according to various embodiments of the present disclosure is shown.

[0019] Figure 3A A computing environment according to various embodiments of the present disclosure is shown, in which a machine learning model of GPU firmware under an improved dynamic voltage and frequency adjustment scheme can be trained.

[0020] Figure 3B An example neural network according to various embodiments of the present disclosure is shown, through which a drawing performance reasoning model, a power consumption reasoning model, and a voltage-frequency reasoning model can be implemented.

[0021] Figure 3C An example drawing instruction for rendering an image frame according to various embodiments of the present disclosure is shown.

[0022] Figure 4 A computing component according to various embodiments of the present disclosure is shown, the computing component including one or more hardware processors and a machine-readable storage medium storing a set of machine-readable / machine-executable instructions. When these instructions are executed, one (or more) hardware processors execute a method to dynamically adjust the working state of a graphics processing unit (GPU).

[0023] The figures depict various embodiments of the disclosed technology for illustration purposes only. The same reference numerals are used in the figures to identify the same elements. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown in the figures may be employed without departing from the principles of the technology disclosed herein. DETAILED DESCRIPTION

[0024] Current implementations of DVFS attempt to adjust the GPU's operating voltage and frequency based on previous workloads to meet the needs of current or subsequent workloads, which is not ideal for power optimization on modern GPUs.

[0025] This article describes an invention that solves the above problems. In various embodiments, a power consumption performance optimization system based on GPU machine learning can be implemented to optimize GPU performance while saving power consumption. Through the power consumption and performance data of the graphics processing unit (GPU) collected from use cases (these use cases use GPU simulation model simulation), the machine learning model associated with the power consumption performance optimization system based on GPU machine learning can be trained to develop various reasoning models to determine the optimal voltage and frequency of the instructions for the GPU to perform work tasks. In general, a work task can be defined as one or more instructions that instruct the GPU to render an image frame. The reasoning model can determine the optimal voltage and frequency instruction by instruction based on the instructions to be executed by the GPU in the work task. Once the reasoning model is trained, the reasoning model can be deployed (i.e., loaded) to the power consumption management firmware of the GPU to intervene in the instruction. Based on each upcoming instruction to be executed by the GPU, the reasoning model in the firmware can determine the optimal voltage and frequency instruction by instruction and provide the optimal voltage and frequency to the GPU to perform each upcoming instruction of the work task. These and other features of the present invention will be further described in detail herein. According to various embodiments of the present disclosure, the optimal voltage and frequency refer to the voltage and frequency that can achieve peak performance at a given power consumption, achieve a given performance at a minimum power consumption, or achieve the best energy efficiency overall.

[0026] Figure 1 A conventional dynamic voltage frequency scaling (DVFS) scheme 100 is shown. Figure 1 As shown, a conventional DVFS scheme 100 may include a computing system 102 configured to render graphics for two-dimensional (2D) or three-dimensional (3D) images or videos. The computing system 102 may include a central processing unit (CPU) 110 coupled to a graphics processing unit (GPU) 120 via a data bus (e.g., a peripheral component interconnect express (PCIe) bus). The CPU 110, together with the GPU 120, may be instructed to render graphics on a display associated with the computing system. For example, a computer application or program (e.g., a virtual reality application, an augmented reality application, etc.) running on the computing system 102 may provide instructions to the CPU 110 and / or the GPU 120 to render graphics associated with the computer program through the display.

[0027] The CPU 110 and the GPU 120 can access the system memory 112. The system memory 112 can store data related to index buffers, shader programs, GPU front-end command processor (FCP) instruction buffers, and parameter buffers required for rendering graphics. The CPU 110 and the GPU 120 can write and / or read various data to the system memory 112. For example, once the GPU 120 performs a graphics rendering function or executes a graphics rendering instruction, such as executing a drawing function, the GPU 120 can write the result data to a memory location on the system memory 112, such as a shader program. Then, the CPU 110 can access the data from the system memory 112 and display the data on a display. In some examples, the CPU 110 has access to an operating system (OS) driver stack 114. The OS driver stack 114 may include various application programming interfaces, graphics drivers, user mode graphics drivers (UMDs), and kernel mode drivers (KMDs). Through OS driver stack 114 , computer applications running on computing system 102 can access CPU 110 and GPU 120 to execute various graphics-related instructions instructed by the computer applications.

[0028] The GPU 120 may include at least one non-volatile memory 122 storing GPU firmware, a power management integrated circuit (PMIC) 124, a phase-locked loop (PLL) 126, and multiple GPU cores 128. The non-volatile memory 122 may be any suitable memory type. For example, the non-volatile memory 122 may be a programmable read-only memory (PROM) or an electrically erasable programmable read-only memory (EEPROM). The GPU firmware may be used to execute instructions based on the work tasks to be performed by the GPU 120 (e.g., Figure 1 The GPU firmware may provide instructions to the PMIC 124, the PLL 126, and the plurality of GPU cores 128. For example, the GPU firmware may receive instructions to render graphics. In this example, the GPU firmware may schedule or assign instructions to one or more of the plurality of GPU cores 128 (e.g., Figure 1 in “Scheduling WL”) and based on previous work tasks (e.g. Figure 1 Executed WL Estimates) estimates the operating voltage and frequency of one or more GPU cores executing one or more scheduled instructions (e.g., Figure 1In some examples, multiple GPU cores 128 may include power consumption and performance counters. The power consumption and performance counters may be configured to count events, such as instructions or the number of instructions executed by the GPU core, the number of pixels / vertices / primitives processed by the GPU core, the number of memory reads / writes performed by the GPU core, etc. In some examples, the power consumption and performance counters may be implemented as registers in hardware, and the main hardware functional blocks of the GPU core may have multiple counters for counting the above-mentioned various events. The values ​​stored in the power consumption and performance counters may represent the work tasks to be processed by the GPU core, and are a direct reflection of the GPU power consumption / performance. For example, the value corresponding to the GPU power consumption stored in the power consumption counter may be used to check whether the power consumption of the GPU exceeds a threshold, such as the maximum power consumption allowed by the GPU core. In this example, if the power consumption exceeds the threshold, it is necessary to quickly adjust the operating voltage and frequency of the GPU core before potential temperature problems occur. In some cases, the counters may record relevant instruction execution events, while the GPU simulation model developed in high-level languages ​​and RTL languages ​​simulates various actual use cases of these instruction execution events when rendering graphics. There may be many variations, and many variations may be considered.

[0029] In some examples, the GPU firmware may include a set of specific instructions for estimating the operating voltage and frequency of multiple GPU cores 128 (e.g., Figure 1 120). This set of specific instructions is responsible for estimating the operating voltage and frequency. For example, machine-compatible code responsible for estimating the operating voltage and frequency of multiple GPU cores 128 can be stored in the GPU firmware for execution. In some examples, PMIC 124 can be logic that is configured to provide operating voltage to multiple GPU cores 128 of GPU 120. In some examples, PMIC 124 can be a separate chip external to GPU 120. In other embodiments, PMIC 124 can be integrated as part of GPU 120. In some examples, PLL 126 can be logic that is configured to provide operating frequency to multiple GPU cores 128 of GPU 120. Similar to PMIC 124, in some examples, PLL 126 can be a separate chip external to GPU 120. In other examples, PLL 126 can be integrated as part of GPU 120.

[0030] Figure 1The conventional DVFS scheme 100 has its disadvantages, for example, the conventional scheme lacks immediate effectiveness. In the conventional method, the operating voltage and frequency provided to the GPU for executing the instruction of the current work task are determined based on the previous work task executed by the GPU and the power consumed by the GPU during the execution of the previous work task. Therefore, the conventional DVFS scheme 100 is reactive in nature. Figure 1 , steps 1 to 4 illustrate the process of how the GPU firmware determines the operating voltage and frequency of the GPU 120 for an upcoming work task based on previous work tasks performed. Step 1: The GPU firmware of GPU 120 receives an instruction ("1Ts-Kick") from the CPU 110 to perform a work task (i.e., render an image frame). Step 2: The GPU firmware schedules the upcoming work task to one or more GPU cores among the multiple GPU cores 128. Step 3: Based on the one or more GPU cores selected to perform the upcoming work task, the GPU firmware estimates the operating voltage and frequency to be provided to the one or more selected GPU cores based on the previous work tasks performed by the one or more selected GPU cores. Step 4: The GPU firmware provides the estimated operating voltage and frequency to the one or more selected GPU cores to perform the upcoming work task. Therefore, the operating voltage and frequency provided to the one or more selected GPU cores are passive for the execution of the upcoming work task.

[0031] Another disadvantage of the traditional DVFS scheme 100 is its coarse granularity. In traditional methods, the GPU's work tasks are usually calculated frame by frame, and the adjustment of the GPU's working voltage and frequency is usually performed frame by frame or multiple frames, which cannot fully optimize the GPU's performance or power consumption. In particular, when processing a frame of an image, hundreds of instructions will be executed, and the power consumption requirements of each instruction may vary. For example, drawing instructions are one of the more important instructions executed by the GPU, especially for three-dimensional graphics rendering. Drawing instruction-related data may include, for example, instruction line argument counts, constant data size, index counts per instance, instance counts, vertex buffer sizes, vertex buffer strides, etc. In some cases, drawing instructions may also have various attributes, such as instruction deadlines, instruction work task cycles, instruction power consumption, instruction frequency / voltage operations, etc. This article will refer to Figure 3CDrawing instructions are discussed in further detail. Since each drawing instruction processes a different amount of data, the power consumption requirements of different drawing instructions may also vary greatly. In three-dimensional graphics rendering, another important instruction executed by the GPU is the shader instruction. Shader instruction related data may include, for example, vertex shaders, subdivision surface control and evaluation shaders, geometry shaders, fragment shaders, and compute shaders. The power consumption requirements of shader instructions may be completely different from those of drawing instructions. Generally, the traditional DVFS scheme 100 is not the best solution, because the traditional DVFS scheme 100 is based on past work tasks, and the resulting operating voltage and frequency do not represent the characteristics of the current work tasks, so the traditional DVFS scheme 100 cannot provide the best energy-saving solution.

[0032] Figure 2 An improved dynamic voltage frequency scaling (DVFS) scheme 200 according to various embodiments of the present disclosure is shown. Figure 1 Similar to the conventional DVFS scheme 100 in FIG. 1 , the improved DVFS scheme 200 may include a computing system 202 configured to render graphics of a two-dimensional or three-dimensional image or video. The computing system 202 includes components similar to Figure 1 The components in the computing system 102 are the same as those in the computing system 102 of FIG. 1 , and therefore, these components will not be discussed again here. Figure 1 Different from the traditional DVFS scheme 100 in the embodiment, the improved DVFS 200 can improve the instant responsiveness or effectiveness of the traditional DVFS scheme 100, can provide DVFS with fine-tuning adjustment granularity so that the optimal operating voltage and frequency can be provided to the GPU core on an instruction-by-instruction basis (rather than on a frame-by-frame basis), and can provide the best solution based on machine learning for GPU power consumption management.

[0033] In some embodiments, the GPU firmware stored in the non-volatile memory 122 of the computing system may include a set of specific instructions (e.g., ML Inference "ML Inference") specifically used to estimate the operating voltage and frequency of multiple GPU cores 128. In some embodiments, this set of specific instructions may be associated with instructions (e.g., code) to execute a machine learning model to "actively" estimate the operating voltage and frequency of multiple GPU cores 128 instruction by instruction. The machine learning model can estimate the operating voltage and frequency of each GPU core in the multiple GPU cores 128 based on the corresponding instructions to be executed by each GPU core in the multiple GPU cores 128. For each instruction to be executed by the GPU core, the machine learning model can estimate the operating voltage and frequency of the GPU core. In this way, the improved DVFS scheme 200 can provide fine-tuning for power consumption optimization of multiple GPU cores 128. In some embodiments, the machine learning model may include multiple inference models. Once multiple inference models are trained, these models can determine the optimal voltage and frequency provided to the GPU core based on the instructions to be executed by the GPU core. The machine learning model and multiple inference models will be discussed in further detail herein with reference to FIG. 3.

[0034] like Figure 2 As shown, the improved DVFS scheme 200 eliminates a step performed by the GPU firmware in the traditional DVFS scheme 100. As described above, in the traditional DVFS scheme 100, the GPU firmware can estimate the operating voltage and frequency of the current work task based on the operating voltage and frequency of the previous work task. In contrast, in the improved DVFS scheme, the GPU firmware can directly estimate the operating voltage and frequency of the current work task by evaluating the instructions of the current work task without evaluating the operating voltage and frequency of the previous work task. Therefore, compared with the traditional DVFS scheme 100, the improved DVFS scheme 200 can respond to fluctuations in work tasks more quickly. By defining the work tasks of the GPU on an instruction-by-instruction basis (e.g., drawing-by-drawing basis), the improved DVFS scheme 200 can adjust the operating voltage and frequency of the GPU in a fine-grained manner, which can significantly improve its performance, because some instructions may require a higher frequency when processing a frame, while some instructions may require a lower frequency when processing the same frame.

[0035] In some embodiments, if the determined power consumption exceeds a threshold value (i.e., power consumption limit, thermal limit, etc.) of the CPU core, the improved DVFS scheme 200 may adjust the operating voltage and frequency provided to the GPU core. For example, the improved DVFS scheme 200 determines the optimal operating voltage and frequency for the GPU core to execute the instruction based on the instruction to be executed by the GPU core. In this example, the optimal operating voltage and frequency of the GPU core will exceed the thermal limit of the GPU core, resulting in thermal protection of the GPU core. In this example, the improved DVFS scheme 200 may adjust the optimal operating voltage and frequency lower to avoid exceeding the thermal limit.

[0036] Figure 3A A computing environment 300 according to various embodiments of the present disclosure is shown, in which a machine learning model of GPU firmware under the improved DVFS solution 200 can be trained. Figure 3A As shown, in some embodiments, the computing environment 300 may include a use case simulation database 302, a machine learning training platform 304, and a trained inference model library 306. The use case simulation database 302 may store various simulation data from the GPU simulation model. These various simulation data may include, for example, simulation instructions of the work task, simulation working voltage, simulation working frequency, simulation power consumption, and simulation performance data of the GPU core when executing the simulation instruction. In some embodiments, the use case simulation database 302 may store power consumption data of the GPU core and its components from real use cases. Using the above data and adopting appropriate algorithms, predictive and adaptive GPU DVFS inference models can be trained under various use case work tasks. The machine learning training platform 304 can be a software development environment that can be used to train the inference model of the machine learning model. For example, in some embodiments, the machine learning training platform 304 can be an integrated development environment (IDE). The trained inference model library 306 can be a database in which the trained inference model of the machine learning model is stored for later access.

[0037] like Figure 3A As shown, data obtained from 3D graphics rendering under various use cases can be obtained, including instructions, instruction execution time, performance and power consumption records, etc. The power consumption data of the 3D graphics rendering task can also be obtained. Then, these data can be used as a training data set to train the reasoning model. In addition, the actual usage data from the simulation of the GPU simulation model can be stored in the use case simulation database 302 and used to train the reasoning model. In some embodiments, the data stored in the use case simulation database 302 can be generated by running a game application on a C language GPU model, and the usage data includes instructions and events at various operating voltages / frequencies, which can be input into the algorithm to train the reasoning model.

[0038] In some embodiments, the machine learning model can be associated with at least three reasoning models: a drawing performance reasoning model, a power consumption reasoning model, and a voltage-frequency reasoning model. The drawing performance reasoning model can be trained based on the simulated or real instructions of the work task, the corresponding voltage / frequency provided to the GPU core, and the execution time of the GPU core when executing the simulated or real instructions of the work task. The power consumption reasoning model can be trained based on the simulated or real instructions of the work task, the corresponding voltage / frequency provided to the GPU core, and the power consumption / performance data of the GPU core when executing the simulated or real instructions of the work task. It can be used to determine the power consumption requirements of various graphics instructions, including drawing instructions, and to ensure whether the power consumption provided to the GPU core exceeds the maximum power consumption allowed by the computing system. The power consumption reasoning model. If exceeded, the operating voltage and frequency need to be adjusted quickly before thermal problems occur. The voltage-frequency reasoning model can be trained based on the simulated or actual instructions of the work task and the performance of the GPU core when executing the simulated or actual instructions of the work task. This article will refer to Figure 3B The drawing performance inference model, power consumption inference model, and voltage-frequency inference model are discussed in further detail.

[0039] Figure 3B An example neural network 340 according to various embodiments of the present disclosure is shown, and the rendering performance reasoning model, the power consumption reasoning model, and the voltage-frequency reasoning model can be implemented by the neural network. In various embodiments, each of the rendering performance reasoning model, the power consumption reasoning model, and the voltage-frequency reasoning model can be implemented as a neural network 340. In general, any suitable neural network can be used to implement the neural network 340. In a specific embodiment, the neural network 340 can be a regression artificial neural network or a convolutional neural network. In the regression artificial neural network, the number of hidden layers and the activation function of the hidden layer can be adjusted during training to optimize the performance of the reasoning model. For example, the neural network 340 is shown to include two hidden layers 342 and 344. In this example, during the training of the neural network 340 of the voltage-frequency reasoning model, the two hidden layers 342, 344 may not be sufficient to accurately determine the optimal operating voltage and frequency of the GPU core. Therefore, in this example, additional hidden layers can be added to the neural network 340 to improve performance. In some embodiments, the neural network 340 may include an error back propagation 346. The error back propagation 346 can help optimize the example neural network 340 during training.

[0040] In some embodiments, drawing instruction data, instruction deadlines, and instruction work task cycles may be used as inputs (e.g., Figure 3B x1, x2, ..., x m) to train the voltage-frequency reasoning model based on the example neural network 340. In some embodiments, the power consumption reasoning based on the example neural network 340 can be trained using drawing instruction data and the operating voltage and frequency associated with the instruction as input.

[0041] As mentioned above, by integrating the trained inference model into the current passive DVFS scheme, a predictive DVFS scheme for adjusting the voltage and frequency of the GPU core can be implemented. Figure 2 The predictive DVFS scheme is discussed in the reference back Figure 2 , note that before the current work task is scheduled for execution, the GPU combined with the inference model can determine and make a request for adjustment of the working voltage and frequency in advance based on the current work task. In this way, predictive DVFS ensures that the current work task can be executed in the best working state, thereby improving the performance / watt index. This is because the inference model is trained to determine the optimal voltage and frequency to be provided to the GPU core before these instructions are executed based on the instructions to be executed during the current work task. Therefore, the present invention is essentially "predictive".

[0042] Figure 3C An example drawing instruction 360 for rendering an image frame according to various embodiments of the present disclosure is shown. Figure 1 As described above, drawing instructions 360 are one of the more important instructions executed by the GPU, especially for three-dimensional graphics rendering. Figure 3C As shown, when executing drawing instructions 360, drawing instructions 360 can be associated with various instruction line parameters. These instruction line parameters may include, for example, instruction line parameter count, instruction line parameter constant data size, instruction line parameter index count by instance, instruction line parameter instance count, instruction line parameter vertex buffer size, instruction line parameter vertex buffer span, etc. In some cases, drawing instructions 360 can be associated with various instruction attributes. These instruction attributes may include, for example, instruction deadline, instruction work task cycle, instruction power consumption, and instruction frequency and voltage. In some embodiments, the instruction line parameters and instruction attributes of various graphics rendering instructions including drawing instructions 360 can be stored in the counter of the GPU core.

[0043] Figure 4 A computing component 400 according to various embodiments of the present disclosure is shown, which includes one or more hardware processors 402 and a machine-readable storage medium 404 storing a set of machine-readable / machine-executable instructions. When these instructions are executed, the one or more hardware processors 402 execute a method to dynamically adjust the working state of a graphics processing unit (GPU).

[0044] In block 406, a machine learning model may be trained to determine the operating voltage and frequency of the GPU core to be provided to the GPU, thereby executing a work task containing multiple instructions. In some embodiments, the machine learning model may include a voltage-frequency reasoning model that is trained to determine the operating voltage and frequency of the GPU core when executing instructions. In some embodiments, the machine learning model may also include a drawing performance reasoning model that is trained to determine the performance data of the GPU core when executing instructions. In some embodiments, the machine learning model may also include a power consumption reasoning model that is trained to determine the power consumption of the GPU core when executing instructions. In some embodiments, the machine learning model may be trained using a first data set generated by a GPU simulation model. The first data set may include data associated with the simulation instructions of the work task, the simulation operating voltage, the simulation operating frequency, the simulation power consumption, and the simulation performance data of the GPU core when executing the simulation instructions. In some embodiments, the instructions of the work task may include at least drawing instructions for rendering three-dimensional graphics.

[0045] In block 408, the trained machine learning model may be deployed to the firmware of the GPU.

[0046] In block 410 , instructions in a work task to be executed by a GPU core may be received.

[0047] In block 412, the trained machine learning model may determine the operating voltage and frequency of the GPU core executing the instruction. In some embodiments, the performance data of the GPU core when executing the instruction may be determined based on the instruction. In some embodiments, the power consumption of the GPU core when executing the instruction may be determined based on the instruction. In some embodiments, the optimal operating voltage and frequency of the GPU core executing the instruction may be determined based on the instruction, the determined performance data, and the determined power consumption. In some embodiments, the optimal operating voltage and frequency of the GPU core executing the instruction may be determined directly based on the instruction.

[0048] Generally, the invention disclosed herein can be implemented using any suitable computing system. For example, in some embodiments, the various inference models disclosed herein can be loaded into a non-volatile memory associated with or accessible by a GPU on a graphics card, such as a PROM or EEPROM. In some embodiments, the inference model disclosed herein can be implemented on an independent computing system or computing device that controls a GPU, for example, a mobile computing device coupled to an external GPU device (eGPU). In these embodiments, the inference model can be stored in a computing system. In various embodiments, a computer system may include a main memory, such as a random access memory (RAM), a cache, and / or other dynamic storage device, which is coupled to a bus for storing information and instructions executed by a processor, such as instructions associated with an inference model. The main memory can also be used to store temporary variables or other intermediate information during the execution of instructions by the processor. When these instructions are stored in a storage medium accessible to the processor, the computer system becomes a dedicated machine that specifically performs the operations specified in the instructions. For example, a computer program or application running on a computer system can control the operating voltage and frequency of the eGPU when rendering graphics.

[0049] In some embodiments, the computer system may also include a read-only memory (ROM) or other static storage device coupled to the bus for storing static information and instructions of the processor. A storage device such as a disk, an optical disk, or a USB flash drive (flash drive) may be provided and coupled to the bus for storing information and instructions.

[0050] In some embodiments, the computer system may be coupled via a bus to an output device, such as a cathode ray tube (CRT) or a liquid crystal display (or touch screen), for displaying information to a user. Input devices, including alphanumeric keys and other keys, may be coupled to the bus for passing information and instruction selections to the processor. Another type of user input device may be a cursor control device, such as a computer mouse or other equivalent input device. The computer system may also include a communication interface, such as a network interface card, coupled to the bus.

[0051] Unless the context requires otherwise, in this specification and claims, the word "comprise" and its variations are to be understood as having an open, inclusive meaning, i.e., "including but not limited to". The recitation of numerical ranges in the specification is intended as a shorthand notation for individually referencing each independent value, where each independent value falls within the range including the values ​​defining the range, and each independent value is incorporated into the specification as if individually recorded herein. In addition, the singular forms "a", "an" and "an" include plural references unless the context clearly dictates otherwise. Phrases such as "at least one of...", "at least one selected from a group of..." or "at least one selected from a group consisting of..." should be interpreted as antonymous conjunctions (for example, should not be interpreted as at least one of A and at least one of B).

[0052] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment in some cases. In addition, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0053] A component implemented as another component may be interpreted as the component functioning in the same or similar manner as the other component and / or the component including the same or similar features, characteristics and parameters as the other component.

Claims

1. A computer-implemented method for dynamically adjusting the working state of a graphics processing unit (GPU), the method comprising: Training a machine learning model to determine an operating voltage and frequency to be provided to a GPU core of the GPU to perform a work task comprising a plurality of instructions; Deploying the trained machine learning model to the firmware of the GPU; Receiving instructions in the work task to be executed by the GPU core; as well as The trained machine learning model is used to determine the operating voltage and frequency of the GPU core executing the instruction.

2. The computer-implemented method of claim 1 , further comprising: Before the GPU core executes the instruction, the determined operating voltage and frequency are provided to the GPU core.

3. The computer-implemented method of claim 1 , wherein: The machine learning model includes a voltage-frequency inference model, which is trained to determine the operating voltage and frequency of the GPU core when executing instructions.

4. The computer-implemented method of claim 3, wherein: The machine learning model also includes a rendering performance inference model, which is trained to determine performance data of the GPU core when executing the instructions.

5. The computer-implemented method of claim 4, wherein: The machine learning model also includes a power consumption reasoning model, which is trained to determine the power consumption of the GPU core when executing the instructions.

6. The computer-implemented method of claim 5, further comprising: If the determined power consumption exceeds a threshold, the operating voltage and frequency of the GPU core are adjusted.

7. The computer-implemented method of claim 5, wherein: The determining the operating voltage and frequency of the GPU core executing the instruction includes: Based on the instruction, determining performance data of the GPU core when executing the instruction; Based on the instruction, determining the power consumption of the GPU core when executing the instruction; and Based on the instruction, the determined performance data and the determined power consumption, an optimal operating voltage and frequency for the GPU core to execute the instruction is generated.

8. The computer-implemented method of claim 1 , further comprising: The machine learning model is trained using a first data set generated by a GPU simulation model, wherein the first data set includes data associated with simulation instructions of the work task, simulation operating voltage, simulation operating frequency, simulation power consumption, and simulation performance data of the GPU core when executing the simulation instructions.

9. The computer-implemented method of claim 1 , further comprising: The machine learning model is trained using a second data set generated by a GPU simulation model, wherein the second data set includes data associated with simulation instructions of the work task, simulation operating voltage, simulation operating frequency, simulation power consumption, and simulation performance data of the GPU core when executing the simulation instructions.

10. The computer-implemented method of claim 1 , further comprising: receiving a second instruction to be executed by the GPU core; as well as Based on the second instruction, a second operating voltage and frequency of the GPU core executing the second instruction are determined through the trained machine learning model.

11. The computer-implemented method of claim 1 , wherein: The instructions of the work task at least include drawing instructions for rendering three-dimensional graphics.

12. The computer-implemented method of claim 1, wherein: The trained machine learning model is not based on a feedback mechanism, in which the trained machine learning model provides the GPU core with an operating frequency and voltage to perform a current work task based on the operating frequency and voltage of the GPU core when performing a previous work task.

13. A non-volatile memory of a computing system, wherein instructions are stored therein, and when the instructions are executed, the computing system executes a method for dynamically adjusting the working state of a graphics processing unit (GPU), the method comprising: Receiving instructions in a work task to be executed by a GPU core of the GPU; as well as The operating voltage and frequency of the GPU core executing the instructions are determined through a machine learning model.

14. The nonvolatile memory according to claim 13, wherein: The method further comprises: Before the GPU core executes the instruction, the determined operating voltage and frequency are provided to the GPU core.

15. The nonvolatile memory according to claim 13, wherein: The machine learning model includes a voltage-frequency inference model, which is trained to determine the operating voltage and frequency of the GPU core when executing instructions.

16. The nonvolatile memory according to claim 15, wherein: The machine learning model also includes a rendering performance inference model, which is trained to determine performance data of the GPU core when executing the instructions.

17. The nonvolatile memory according to claim 16, wherein: The machine learning model also includes a power consumption reasoning model, which is trained to determine the power consumption of the GPU core when executing the instructions.

18. The nonvolatile memory according to claim 17, wherein: When the instructions are executed, the computing system is caused to further execute: If the determined power consumption exceeds a threshold, the operating voltage and frequency of the GPU core are adjusted.

19. The nonvolatile memory according to claim 17, wherein: The determining the operating voltage and frequency of the GPU core executing the instruction includes: Based on the instruction, determining performance data of the GPU core when executing the instruction; Based on the instruction, determining the power consumption of the GPU core when executing the instruction; and Based on the instruction, the determined performance data and the determined power consumption, an optimal operating voltage and frequency for the GPU core to execute the instruction is generated.

20. The nonvolatile memory according to claim 13, wherein: The instructions of the work task at least include drawing instructions for rendering three-dimensional graphics.

Images