GPU graphics processor power optimization scheduling method for tablet computers

Abstract

This invention relates to the field of graphics processing unit (GPU) technology, specifically to a power consumption optimization scheduling method for GPUs used in tablet computers. The method obtains the vertex throughput and triangle culling rate output from the vertex processing stage of the current frame, inputs them into a Long Short-Term Memory (LSTM) network, and outputs the predicted computational load for the rasterization and pixel shading stages. The hidden layer state of the LSM network is updated based on the actual pixel shading stage time of the previous frame. A pre-built voltage-frequency lookup table is consulted based on the predicted computational load to complete the configuration change of the GPU core voltage and frequency before the rasterization stage of the current frame begins. This invention eliminates the adjustment lag caused by load feedback in the latter half of frame rendering, avoids the invalid power consumption caused by maintaining high frequency operation in low-load rendering frames, and reduces the dynamic power consumption fluctuations of the tablet computer's GPU during continuous rendering.

Description

Technical Field

[0001] This invention relates to the field of graphics processor technology, and more specifically to a GPU graphics processor power optimization scheduling method for tablet computers. Background Technology

[0002] In the field of power consumption scheduling for tablet PC graphics processors, existing conventional solutions typically read the actual execution time recorded by an internal hardware counter after the graphics processor completes all rendering operations for the current frame or the pixel shading stage. Based on this actual execution time, the system adjusts the core power supply voltage and operating frequency of the graphics processor before the next frame's rendering begins. This method, which relies on hardware feedback data after the completion of a full frame, depends entirely on historical rendering data and is a type of hysteresis closed-loop control.

[0003] Based on the specific implementation of the aforementioned conventional technical solutions, since the actual execution time on which the adjustment is based can only be obtained after the current frame rendering is completed, and the voltage and frequency change instructions can only take effect in the next frame, there is an adjustment delay of at least one frame. When the complexity of the 3D scene running on the tablet computer changes abruptly between adjacent frames, this adjustment delay prevents the graphics processor from obtaining a voltage and frequency configuration that matches the actual load during the rasterization and pixel shading stages of the current frame. Consequently, it maintains an excessively high frequency during the subsequent processing stages of low-load rendering frames, resulting in ineffective power consumption. Summary of the Invention

[0004] The purpose of this invention is to provide a GPU graphics processor power optimization scheduling method for tablet computers, which can solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A GPU power consumption optimization scheduling method for tablet computers includes: obtaining the vertex throughput and triangle culling rate output by the vertex processing stage of the current frame of the tablet computer's GPU; inputting the vertex throughput and triangle culling rate into a frame load prediction model, and outputting the predicted computational load of the rasterization and pixel shading stages of the current frame, wherein the frame load prediction model is a Long Short-Term Memory (LSTM) network, and the hidden layer state of the LSM network is updated based on the actual pixel shading stage time of the previous frame; querying a pre-built voltage-frequency lookup table according to the predicted computational load, and completing the configuration change of the GPU core voltage and frequency before the rasterization stage of the current frame starts; the weight matrices of the input gate, forget gate, and output gate of the LSM network are offline trained and solidified during the tablet computer system initialization stage based on the vertex throughput, triangle culling rate, and actual pixel shading time of historical rendering frames.

[0006] Preferably, obtaining the vertex throughput and triangle culling rate output by the vertex processing stage of the current frame of the tablet computer's graphics processor includes: when the current frame enters the vertex shader execution stage, activating a hardware performance counter configured in the vertex processing pipeline of the graphics processor; the hardware performance counter accumulates the number of vertices passing through the vertex shader to obtain the vertex throughput, and accumulates the number of triangles culled by backface culling and view frustum clipping to obtain the total number of culled triangles; at the end of the vertex processing stage, reading the number of vertices and the total number of culled triangles from the hardware performance counter, and using the ratio of the total number of culled triangles to the initial total number of triangles before entering the vertex processing stage as the triangle culling rate.

[0007] Preferably, the hidden layer state of the Long Short-Term Memory (LSTM) network is updated based on the actual pixel shading stage time of the previous frame, including: during the execution of the pixel shading stage of the previous frame, recording the actual pixel shading stage time using a periodic precision timer configured at the end of the pixel pipeline of the graphics processor; before the vertex processing stage of the current frame begins, using the difference between the actual pixel shading stage time and the prediction computation load of the previous frame as an error feature vector, concatenating the error feature vector with the hidden layer state vector output by the LSM network in the previous frame, inputting the concatenated vector into the forget gate of the LSM network, and performing element-wise multiplication on the hidden layer state vector of the previous frame through the output vector of the forget gate to obtain the updated hidden layer state of the current frame.

[0008] Preferably, querying a pre-built voltage frequency lookup table based on the predicted computational load includes: the voltage frequency lookup table uses discretized computational load levels as indexes to store the core clock frequency value and supply voltage value uniquely corresponding to the computational load level; the voltage frequency lookup table also stores the voltage stabilization time corresponding to the core clock frequency value; mapping the predicted computational load to the closest computational load level in the voltage frequency lookup table, extracting the corresponding core clock frequency value, supply voltage value, and voltage stabilization time; calculating the idle time window between the end time of the vertex processing stage of the current frame and the start time of the rasterization stage; when the idle time window is greater than the voltage stabilization time, triggering the instruction to issue the supply voltage value and the core clock frequency value.

[0009] Preferably, before the rasterization stage of the current frame starts, the configuration change of the graphics processor core voltage and frequency is completed, including: when the idle time window is longer than the voltage stabilization time, sending a voltage adjustment command containing the supply voltage value to the power management integrated circuit of the tablet computer to drive the power management integrated circuit to adjust the output of the buck converter connected to the power rail of the graphics processor core; after detecting that the feedback voltage of the graphics processor core power rail reaches the supply voltage value, sending a frequency adjustment command containing the core clock frequency value to the clock generator to control the clock generator to update the pixel shader clock signal output to the graphics processor; after the frequency of the clock signal is locked, sending a release signal to the rasterization execution pipeline of the graphics processor to start the rasterization stage.

[0010] Preferably, the weight matrices of the input gate, forget gate, and output gate of the Long Short-Term Memory network are offline trained and solidified during the tablet computer system initialization phase based on the vertex throughput, triangle culling rate, and actual pixel shading time of historical rendering frames. This includes: during the tablet computer system initialization phase, traversing multiple preset benchmark 3D rendering test scenes, collecting the vertex throughput and triangle culling rate of each frame in the benchmark 3D rendering test scene as input samples, and collecting the actual pixel shading time of the corresponding frame as label samples; using mean squared error as the loss function, iteratively training the initial model of the Long Short-Term Memory network containing the input gate, forget gate, and output gate using the backpropagation algorithm; converting the weight matrix corresponding to the iteration result that satisfies the convergence condition of the loss function into fixed-point numbers, and burning the fixed-point numbers into the read-only memory of the microcontroller of the graphics processor.

[0011] Preferably, after reading the number of vertices and the total number of culled triangles from the hardware performance counter, the method further includes: comparing the number of vertices with a preset upper limit threshold for the number of vertices; when the number of vertices exceeds the upper limit threshold, marking the vertex throughput as an abnormal state and replacing the vertex throughput of the abnormal state with the vertex throughput of the previous frame; comparing the triangle culling rate with a preset lower limit threshold for the culling rate; when the triangle culling rate is lower than the lower limit threshold, marking the triangle culling rate as an abnormal state and replacing the triangle culling rate of the abnormal state with the triangle culling rate of the previous frame; and inputting the currently read vertex throughput and triangle culling rate into the frame load prediction model only when neither the vertex throughput nor the triangle culling rate is marked as an abnormal state.

[0012] Preferably, the difference between the actual pixel coloring stage time and the prediction computation load of the previous frame is used as the error feature vector, including: obtaining the standard pixel processing cycle number corresponding to the prediction computation load of the previous frame, and converting the actual pixel coloring stage time into the actual pixel processing cycle number; calculating the absolute value of the difference between the actual pixel processing cycle number and the standard pixel processing cycle number; extracting the baseline load value corresponding to the computation load level of the prediction computation load of the previous frame, calculating the ratio of the absolute value of the difference to the baseline load value, and using the ratio as a dynamic scaling factor; using the result of multiplying the absolute value of the difference by the dynamic scaling factor as the scalar value of the error feature vector, and filling the scalar value into a zero vector of a preset dimension to form the error feature vector.

[0013] Preferably, after calculating the idle time window between the end time of the vertex processing stage of the current frame and the start time of the rasterization stage, the method further includes: when the idle time window is less than or equal to the voltage stabilization time, querying the voltage frequency lookup table for at least one low load level lower than the computational load level corresponding to the predicted computational load, extracting the highest low load level among the at least one low load level whose voltage stabilization time is less than the idle time window; extracting the degraded supply voltage value and degraded core clock frequency value corresponding to the highest low load level, sending a voltage adjustment command containing the degraded supply voltage value to the power management integrated circuit, and after the feedback voltage of the graphics processor core power rail reaches the degraded supply voltage value, sending a frequency adjustment command containing the degraded core clock frequency value to the clock generator, and sending the release signal to the rasterization execution pipeline after the frequency of the clock signal is locked.

[0014] Preferably, after burning the fixed-point numbers into the read-only memory of the microcontroller of the graphics processor, the method further includes: during the vertex processing stage of the current frame, the microcontroller reads the fixed-point numbers of the weight matrices corresponding to the input gate, the forget gate, and the output gate from the read-only memory; quantizes and truncates the vertex throughput and the triangle culling rate respectively, converting them into input fixed-point numbers with a set bit width; the microcontroller performs matrix multiplication using the input fixed-point numbers and the fixed-point numbers of the weight matrix through shift and addition operations, performs dequantization floating-point conversion on the result of the matrix multiplication through a preset lookup table, and uses the converted floating-point number as the prediction calculation load output by the frame load prediction model.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention obtains the vertex throughput and triangle culling rate of the current frame's vertex processing stage, inputs them into a Long Short-Term Memory (LSTM) network that updates the hidden layer state using the actual pixel shading time of the previous frame, outputs the prediction computation load for subsequent rasterization and pixel shading stages, and completes the core voltage and frequency configuration changes by querying a lookup table before the rasterization stage begins. This technical approach changes the technical path that relies on feedback from the entire frame rendering process, moving load prediction and frequency adjustment actions forward to before the rasterization stage. This eliminates the voltage and frequency adjustment lag caused by feedback in the latter half of frame rendering, avoids maintaining high frequency operation in the subsequent processing stages of low-load rendered frames, and reduces the dynamic power consumption fluctuations of the graphics processor during continuous rendering.

[0016] 2. By using hardware performance counters to count the number of vertices and the total number of triangles removed to obtain pre-order features, an error feature vector consisting of the difference between actual processing time and predicted load is introduced into the Long Short-Term Memory (LSTM) network and concatenated with the hidden layer state of the previous frame to update the forget gate, maintaining the continuity of the predicted state during inter-frame load abrupt changes. A voltage frequency lookup table introduces a comparison mechanism between voltage stabilization time and idle time windows. When idle time is insufficient, it automatically downgrades to match the highest or lowest load level that meets the constraints of the stabilization time, ensuring the physical timing requirements of the voltage adjustment process. Model weights are converted to fixed-point numbers through offline training and burned into read-only memory. The microcontroller performs matrix multiplication and table lookup dequantization through shift and addition operations, reducing the computational resource consumption and inference time of the prediction process, enabling pre-prediction and scheduling changes to be completed within the time gap between the vertex processing stage and the rasterization stage. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall execution process of GPU power consumption optimization scheduling in this invention. Figure 2 This is a flowchart of the hardware acquisition process for vertex throughput and triangle culling rate in this invention; Figure 3 This is a flowchart of the vertex data anomaly detection and correction process of the present invention; Figure 4 This is a flowchart of the LSTM hidden layer state error update process of the present invention; Figure 5 This is a flowchart of the GPU core voltage frequency configuration and timing verification process of the present invention; Figure 6 This is a flowchart of the offline training and weight fixing process of the LSTM model of the present invention. Detailed Implementation

[0018] refer to Figure 1In one embodiment, based on the system-on-a-chip architecture of a tablet computer, the graphics processor is integrated inside the system-on-a-chip and interconnected with the power management integrated circuit and the clock generator through the system-on-a-chip bus. The rendering pipeline of the graphics processor follows the pipeline architecture of the standard graphics rendering API and is divided into a vertex processing stage, a rasterization stage, a pixel shading stage and a frame output stage. There is a fixed instruction synchronization gap between the vertex processing stage and the rasterization stage, which provides an executable time window for voltage and frequency configuration changes.

[0019] Specifically, after the rendering process of the current frame starts, the front-end pipeline of the graphics processor first executes the relevant operations of the vertex processing stage. After the vertex processing stage is completed, the vertex throughput and triangle culling rate of the stage are obtained. Vertex throughput represents the number of vertices that are shaded and output to subsequent pipelines per unit time in the vertex processing stage, and triangle culling rate represents the ratio of the number of triangles culled after backface culling and view frustum clipping operations in the vertex processing stage to the initial total number of triangles entering the vertex processing stage.

[0020] The acquired vertex throughput and triangle culling rate are normalized to form a 2×1 input vector, which is then input into the frame load prediction model. The frame load prediction model employs a Long Short-Term Memory (LSTM) network architecture. The LSM network has an input dimension of 2, a hidden layer dimension of 16, and an output dimension of 1. The output value represents the predicted computational load for the rasterization and pixel shading stages of the current frame. The predicted computational load is quantized by the number of standard clock cycles required to complete the rasterization and pixel shading stages.

[0021] The gating operation of the Long Short-Term Memory (LSTM) network is implemented using the following formula, where t represents the temporal node corresponding to the current frame, and t-1 represents the temporal node corresponding to the previous frame: The formula for the forget gate is: in, This is the output vector of the forget gate in the current frame. It is a sigmoid activation function, with an output range of 0 to 1. Here is the weight matrix for the forget gate. This is the hidden layer state vector output from the previous frame. The input vector for the current frame consists of vertex throughput and triangle culling rate. This is the bias term for the forget gate.

[0022] The formula for input gate operation is: in, The output vector of the input gate in the current frame. Here is the weight matrix of the input gate. This is the bias term for the input gate.

[0023] The formula for calculating candidate cell states is: in, This is the candidate vector of cell states for the current frame. It is a hyperbolic tangent activation function with an output range of -1 to 1. This is the weight matrix for cell states. This is a bias term for the cell state.

[0024] The formula for updating cell state is: in, This is the cell state vector for the current frame. This is an element-wise multiplication operation. This is the cell state vector from the previous frame.

[0025] The formula for the output gate is: in, This is the output vector of the output gate in the current frame. Here is the weight matrix of the output gate. This is the bias term for the output gate.

[0026] The output formula for the hidden layer state is: in, This is the hidden layer state vector output for the current frame, used for hidden layer state updates and model inference in the next frame.

[0027] The output formula for predicting the load is: in, The prediction calculation load is calculated for the current frame output. This is the weight matrix of the output layer. This is the bias term for the output layer.

[0028] The hidden state of the Long Short-Term Memory (LSTM) network is updated based on the actual pixel shading stage time of the previous frame. Specifically, after the pixel shading stage of the previous frame is completed, the actual execution time of that stage is collected, converted into standard clock cycles, and compared with the prediction computation load output by the previous frame. An error correction term is generated based on the comparison result, and the error correction term is fused with the hidden state vector output by the previous frame to complete the initialization update of the LSM network hidden state of the current frame. This ensures that the hidden state can carry the load prediction error information between frames, thereby improving the stability of load prediction during continuous frame rendering.

[0029] After obtaining the predicted computational load, a pre-built voltage-frequency lookup table is queried based on this predicted computational load. The voltage-frequency lookup table is indexed by discrete computational load levels. Each computational load level corresponds to a unique continuous computational load interval. The predicted computational load within each interval is mapped to that level. Each level stores the graphics processor core clock frequency value, the supply voltage value, and the voltage stabilization time corresponding to that supply voltage value that matches the load interval. The voltage stabilization time is the maximum time required for the power management integrated circuit to adjust the output voltage to the target supply voltage value and complete the stabilization lock.

[0030] Table 1 Graphics Processor Core Voltage Frequency Lookup Table and Basic Parameter Mapping Table The computational load levels in Table 1 correspond to the rendering load capabilities of the graphics processor. The core clock frequency and power supply voltage values ​​for each level are determined through hardware characteristic testing of the graphics processor to ensure that rendering calculations can be completed stably within the corresponding load range. Meanwhile, the voltage stabilization time is obtained based on actual measurements of the hardware characteristics of the power management integrated circuit, covering the voltage adjustment time under the worst operating conditions.

[0031] Based on the queried parameters, the configuration changes of the graphics processor core voltage and frequency are completed before the rasterization stage of the current frame begins. Specifically, the time interval between the actual end time of the vertex processing stage of the current frame and the preset start time of the rasterization stage is calculated. This time interval is used as an idle time window. The idle time window is compared with the queried voltage stabilization time. When the idle time window is greater than the voltage stabilization time, the corresponding voltage adjustment command and frequency adjustment command are generated. First, the voltage adjustment command is sent to the power management integrated circuit. After the power supply voltage stabilizes, the frequency adjustment command is sent to the clock generator. After the clock frequency is locked, a start command is sent to the graphics processor's rasterization pipeline to ensure that the voltage and frequency configuration changes are completed before the rasterization stage begins.

[0032] The weight matrices of the input, forget, and output gates of the Long Short-Term Memory (LSTM) network are trained and solidified offline during the tablet system initialization phase. Specifically, during the initialization phase after the tablet is first powered on or after a system restart, the system calls a pre-set training dataset. This dataset contains the vertex throughput, triangle culling rate, and corresponding actual pixel shading time of historical rendering frames. Using mean squared error as the loss function, the backpropagation algorithm is used to iteratively train the initial model of the LSM network. When the decrease in the loss function is lower than a preset convergence threshold, training stops, and the weight matrices and bias terms corresponding to the input, forget, and output gates obtained after training are solidified and stored in the on-chip memory of the graphics processor. In subsequent rendering processes, the frame load prediction model directly calls the solidified weight matrices to perform inference operations without needing to update the weight parameters online.

[0033] This embodiment performs load prediction based on the output characteristics of the vertex processing stage of the current frame after the vertex processing stage is completed and before the rasterization stage is started, and completes the voltage and frequency configuration changes. The execution timing of load scheduling is moved forward from after the frame rendering ends to the rendering process of the current frame, eliminating the scheduling adjustment delay between frames, avoiding the graphics processor from maintaining an excessively high voltage and frequency configuration during the rendering process of low-load frames, and reducing the dynamic power consumption of the graphics processor during continuous rendering.

[0034] refer to Figure 2 In a preferred embodiment, the vertex throughput and triangle culling rate are obtained through hardware performance counters configured in the vertex processing pipeline of the graphics processor. Specifically, the vertex processing pipeline of the graphics processor includes a vertex shader unit, a triangle assembly unit, a backface culling unit, and a view frustum culling unit. Hardware performance counters are respectively configured at the outputs of the vertex shader unit, the backface culling unit, and the view frustum culling unit. Each hardware performance counter is synchronized with the vertex processing pipeline clock of the graphics processor and has periodic-level counting accuracy.

[0035] When the current frame enters the vertex shader execution phase, all configured hardware performance counters are synchronously started and their count values ​​are reset to zero. The hardware performance counters configured at the vertex shader unit outputs accumulate the number of vertices that have completed shading processing by the vertex shader in each clock cycle. At the end of the vertex processing phase, the accumulated count value of this counter is the total number of vertices that have passed through the vertex shader. Based on this total number of vertices and the actual execution time of the vertex processing phase, the vertex throughput is calculated.

[0036] The formula for calculating vertex throughput is: in, The vertex throughput of the current frame. The total number of vertices that pass through the vertex shader. This represents the actual execution time of the vertex processing phase in the current frame.

[0037] A hardware performance counter configured at the output of the backface culling unit accumulates the number of triangles determined to be invisible by the backface culling operation in each clock cycle. A hardware performance counter configured at the output of the view frustum clipping unit accumulates the number of triangles determined to be outside the view frustum by the view frustum clipping operation in each clock cycle. At the end of the vertex processing phase, the accumulated count values ​​of the two counters are read, summed to obtain the total number of culled triangles. The ratio of the total number of culled triangles to the initial total number of triangles before entering the vertex processing phase is used as the triangle culling rate.

[0038] The formula for calculating the triangle rejection rate is: in, The triangle culling rate for the current frame. This represents the total number of triangles removed in the current frame. This represents the initial total number of triangles before the current frame enters the vertex processing stage.

[0039] Table 2 Hardware Performance Counter Parameter Configuration Table for Vertex Processing Stage 0 Vertex shader unit output Number of vertices processed by coloring Vertex coloring completion signal valid 32 Accumulates per clock cycle 1 Backside rejection unit output Number of triangles removed from the back Backside rejection signal valid 32 Accumulates per clock cycle 2 Output of the frustum clipping unit Number of triangles clipped by the view frustum The cone clipping determination signal is valid. 32 Accumulates per clock cycle 3 Vertex processing stage control unit Clock cycles during vertex processing The valid signal during the vertex processing stage is high. 32 Accumulates per clock cycle The hardware performance counters in Table 2 are all driven by the graphics processor's pipeline clock. The counting results have no software processing delay and can be read within the first clock cycle after the vertex processing stage ends, ensuring that the input data for load prediction can be obtained in a timely manner. The count value of counter 3 is used to calculate the actual execution time of the vertex processing stage, providing a time reference for the calculation of vertex throughput.

[0040] refer to Figure 3 Furthermore, after reading the number of vertices and the total number of triangles culled from the hardware performance counters, the read values ​​undergo anomaly detection and correction processing. Specifically, a vertex count upper limit threshold is preset, which is determined based on the maximum vertex processing capability of the graphics processor's vertex processing pipeline. The number of vertices read through the vertex shader is compared with the vertex count upper limit threshold. When the number of vertices exceeds the vertex count upper limit threshold, the vertex throughput obtained in this instance is marked as an anomaly, and the vertex throughput obtained during the rendering process of the previous frame that was not marked as an anomaly is used as the vertex throughput input value for the current frame.

[0041] Meanwhile, a lower threshold for the culling rate is preset. This threshold is determined based on the minimum visible triangle ratio of the 3D rendering scene. The calculated triangle culling rate is compared with the lower threshold. When the triangle culling rate is lower than the lower threshold, the triangle culling rate obtained this time is marked as an abnormal state. The triangle culling rate obtained in the previous frame rendering process that was not marked as an abnormal state is used as the triangle culling rate input value for the current frame.

[0042] Only when neither vertex throughput nor triangle culling rate is marked as abnormal will the currently read and calculated vertex throughput and triangle culling rate be input into the frame load prediction model to perform subsequent load prediction operations.

[0043] This embodiment achieves periodic, precise acquisition of vertex throughput and triangle culling rate through a hardware performance counter, eliminating the latency and errors caused by software statistics. At the same time, through an outlier detection and correction mechanism, it avoids load prediction input errors caused by hardware counter failures or pipeline data anomalies, thereby improving the stability and reliability of load prediction results.

[0044] refer to Figure 4 In another preferred embodiment, the hidden layer state of the Long Short-Term Memory (LSTM) network is updated based on the actual pixel shading stage time of the previous frame. Specifically, a periodic precision timer is configured at the end of the GPU's pixel pipeline. This timer is synchronized with the GPU's pixel shader clock, with a timing precision of a single pixel shader clock cycle. During the execution of the pixel shading stage of the previous frame, when the start instruction for the pixel shading stage takes effect, the periodic precision timer is started synchronously. When the last pixel shading task of the pixel shading stage is completed and the frame buffer write instruction takes effect, the periodic precision timer is stopped. The number of clock cycles recorded by the timer is the actual pixel shading stage time of the previous frame.

[0045] Before the vertex processing stage of the current frame begins, the hidden layer state update operation of the Long Short-Term Memory network is completed. Specifically, the prediction computation load output by the frame load prediction model of the previous frame is obtained. This prediction computation load corresponds to the standard pixel processing cycle number of the rasterization and pixel coloring stage of the previous frame. The actual pixel coloring stage time of the previous frame is converted into the actual pixel processing cycle number of the same dimension as the standard pixel processing cycle number. The absolute value of the difference between the actual pixel processing cycle number and the standard pixel processing cycle number is calculated, and an error feature vector is generated based on the absolute value of the difference.

[0046] The formula for calculating the dynamic scaling factor is: in, This is the dynamic scaling factor corresponding to the current frame. This represents the actual number of pixel processing cycles in the previous frame. This represents the number of standard pixel processing cycles corresponding to the prediction load of the previous frame. The baseline load value is the value corresponding to the load level of the prediction calculation load in the previous frame. The baseline load value is the median value of the standard cycle number range corresponding to the load level.

[0047] The formula for calculating the scalar value of the error eigenvector is: in, denoted as the scalar value of the error eigenvector.

[0048] The calculated scalar values ​​are filled into a zero vector of a preset dimension to form an error feature vector with the same dimension as the hidden layer state vector. The preset dimension is the same as the hidden layer dimension of the Long Short-Term Memory network, which is 16 dimensions in this embodiment.

[0049] The error feature vector is concatenated with the hidden layer state vector output by the Long Short-Term Memory network in the previous frame. The formula for calculating the concatenated vector is as follows: in, The concatenated input vector has a dimension of 32×1.

[0050] The concatenated vector is input into the forget gate of the Long Short-Term Memory (LSTM) network. The sigmoid activation function of the forget gate outputs a weight vector in the range of 0-1. Each element of this weight vector corresponds to the retained weight of the corresponding element in the hidden layer state vector of the previous frame. The output vector of the forget gate is multiplied element-wise with the hidden layer state vector of the previous frame to obtain the updated initial value of the hidden layer state of the current frame. At the same time, the cell state is synchronously updated based on this concatenated vector, ensuring that the internal state of the LSM network can carry the prediction error information between frames and achieve a smooth transition of the hidden layer state when the load of adjacent frames changes abruptly.

[0051] Table 3. Mapping Table of Error Feature Vectors and Hidden Layer State Update Parameters The parameters in Table 3 correspond to the calculated load levels in the voltage frequency lookup table. Based on the prediction error fluctuation characteristics of different load ranges, differentiated dynamic scaling factor ranges and weight thresholds are set. The correction of prediction error is greater in the low load range, and the stability of the hidden layer state is maintained in the high load range. The hidden layer state update coefficient is used to control the influence weight of the error feature vector on the hidden layer state, so as to avoid the hidden layer state from changing drastically due to the error value being too large.

[0052] Furthermore, during the load prediction inference process of the current frame, the Long Short-Term Memory network performs complete gating operations and state updates based on the updated initial values ​​of the hidden layer states, combined with the vertex throughput and triangle culling rate vectors input to the current frame, outputs the prediction computation load of the current frame, and stores the hidden layer states and cell states output by this operation into on-chip registers for use in the hidden layer state update operation of the next frame.

[0053] This embodiment introduces an error feature vector based on the difference between actual time consumption and predicted load, integrating the prediction error information between frames into the hidden layer state update process of the Long Short-Term Memory network. This optimizes the state filtering mechanism of the forget gate, avoids the jump of hidden layer state when the load changes suddenly in adjacent frames, and improves the prediction accuracy and adaptability of the load prediction model in dynamic rendering scenarios.

[0054] refer to Figure 5 In another preferred embodiment, the process of querying the pre-built voltage-frequency lookup table based on the predicted computational load is implemented using a combination of load level mapping and constraint verification. Specifically, the voltage-frequency lookup table uses discrete computational load levels as indexes, with each load level corresponding to a unique continuous computational load interval. Each interval stores the core clock frequency value, the power supply voltage value, and the voltage stabilization time corresponding to the power supply voltage value that match that load interval. The core clock frequency value and the power supply voltage value satisfy the voltage-frequency scaling curve of the graphics processor, ensuring that the voltage and frequency combination can support the rendering computation requirements of the corresponding load interval while maintaining the stable operation of the graphics processor.

[0055] The predicted computational load is mapped to the closest computational load level in the voltage-frequency lookup table. Specifically, when the predicted computational load is at the boundary between two adjacent computational load levels, it is mapped upwards to a higher computational load level, and the core clock frequency, supply voltage, and voltage stabilization time corresponding to that level are extracted. The idle time window between the end of the vertex processing phase and the start of the rasterization phase in the current frame is calculated using the following formula: in, For idle time windows, This is the preset start time for the rasterization stage. This is the actual end time of the vertex processing phase.

[0056] The idle time window is compared with the extracted voltage stabilization time. When the idle time window is longer than the voltage stabilization time, a command to send the supply voltage value and the core clock frequency value is triggered. Specifically, a voltage adjustment command containing the target supply voltage value is first sent to the power management integrated circuit of the tablet computer. After receiving the command, the power management integrated circuit adjusts the output duty cycle of the buck converter connected to the core power rail of the graphics processor, driving the output voltage to change towards the target supply voltage value. At the same time, the power management integrated circuit collects the actual voltage value of the core power rail in real time through the feedback pin. When the actual voltage value reaches the target supply voltage value and the fluctuation range is within the preset stabilization threshold range, a voltage stabilization confirmation signal is sent to the microcontroller of the graphics processor.

[0057] After receiving the voltage stabilization confirmation signal, the microcontroller sends a frequency adjustment command containing the target core clock frequency value to the clock generator. After receiving the command, the clock generator adjusts the frequency division coefficient of the internal phase-locked loop and updates the pixel shader clock signal output to the graphics processor. When the phase-locked loop completes frequency locking, it sends a frequency locking confirmation signal to the microcontroller. After receiving the signal, the microcontroller sends a release signal to the rasterization execution pipeline of the graphics processor to start the execution of the rasterization stage.

[0058] Furthermore, when the idle time window is less than or equal to the voltage stabilization time, a load level degradation matching operation is performed. Specifically, the voltage frequency lookup table is used to query all low load levels below the calculated load level corresponding to the predicted calculated load. The voltage stabilization time corresponding to each low load level is extracted sequentially. Low load levels with voltage stabilization times less than the idle time window are selected. Among the selected low load levels, the highest low load level is selected as the target degradation level. The degradation supply voltage value and degradation core clock frequency value corresponding to the target degradation level are extracted. Following the same instruction issuance process as described above, a voltage adjustment instruction is first sent to the power management integrated circuit. After the voltage stabilizes, a frequency adjustment instruction is sent. After the frequency is locked, a rasterization release signal is sent.

[0059] refer to Figure 6 In this embodiment, the weight matrices of the input gate, forget gate, and output gate of the Long Short-Term Memory network are trained and solidified offline during the tablet computer system initialization phase based on the vertex throughput, triangle culling rate, and actual pixel shading time of historical rendering frames. Specifically, during the tablet computer system initialization phase, multiple preset benchmark 3D rendering test scenes are traversed, including different types of rendering scenes such as static scenes, dynamic scenes, high-polygon scenes, and low-polygon scenes. During the continuous rendering process of each benchmark 3D rendering test scene, the vertex throughput and triangle culling rate of each frame are collected as input samples, and the actual pixel shading time of the corresponding frame is collected as label samples to construct a training dataset.

[0060] Using the mean squared error as the loss function, the formula for calculating the loss function is: in, Let the mean squared error loss function be . The total number of training samples, For the first The predicted loading value for each sample, For the first The label value of each sample is the standard number of cycles after the conversion of the actual pixel coloring time of the corresponding frame.

[0061] The backpropagation algorithm is used to iteratively train the initial model of the Long Short-Term Memory Network, which includes an input gate, a forget gate, and an output gate. In each iteration, the input samples of the training dataset are input into the initial model to obtain the predicted output value. The loss value between the predicted output value and the labeled sample is calculated. Based on the loss value, the gradient of the weight matrix and the bias term is calculated using the chain rule. The gradient descent algorithm is used to update the values ​​of the weight matrix and the bias term. When the decrease of the loss function in multiple consecutive iterations is lower than the preset convergence threshold, the model is determined to meet the convergence condition, and the iterative training is stopped.

[0062] The weight matrix and bias terms corresponding to the iteration results that meet the convergence condition are converted into fixed-point numbers. The fixed-point number conversion adopts the Q-format notation method, and the conversion formula is as follows: in, For numerical values The corresponding fixed-point number, For decimal places, Integer width For rounding operations, This is a sign function; it outputs 0 for positive numbers and 1 for negative numbers.

[0063] The converted fixed-point values ​​are burned into the read-only memory of the graphics processor's microcontroller. During subsequent rendering, the microcontroller directly reads the solidified fixed-point weights from the read-only memory, eliminating the need for online training and floating-point operations.

[0064] Furthermore, after the vertex processing stage of the current frame is completed, the microcontroller reads the fixed-point numbers of the weight matrices corresponding to the input gate, forget gate, and output gate from the read-only memory. It then quantizes and truncates the vertex throughput and triangle culling rate of the current frame, converting them into input fixed-point numbers with the same bit width as the weight matrix. The microcontroller performs matrix multiplication using shift and addition operations, utilizing the input fixed-point numbers and the fixed-point numbers of the weight matrix. This avoids the computational latency and resource consumption caused by floating-point multiplication. The result of the matrix multiplication is converted to a floating-point number through a preset inverse quantization lookup table. The converted floating-point number is used as the prediction calculation load output by the frame load prediction model to perform subsequent voltage frequency lookup and configuration operations.

[0065] Table 4. Fixed-point parameters of the weight matrix of the Long Short-Term Memory network The fixed-point quantization parameters in Table 4 are determined based on the numerical distribution range of the weight matrix, ensuring that the quantized fixed-point number can cover the entire range of weight values, while controlling the quantization error within a preset threshold range to avoid loss of prediction accuracy caused by fixed-point quantization. The allocation of integer bit width and decimal bit width is determined based on the dynamic range of the weight values, maximizing numerical accuracy while ensuring no overflow.

[0066] This embodiment achieves adaptive degradation matching of voltage frequency configuration by constraining the idle time window and voltage stability establishment time, ensuring the timing compliance of configuration change operations. At the same time, by offline training of the weight matrix, fixed-point transformation and read-only memory solidification, combined with the fixed-point number operation of the microcontroller, the computation time and resource consumption of the load prediction inference process are reduced, ensuring that the entire prediction and scheduling process can be completed within the time gap between the vertex processing stage and the rasterization stage.

Claims

1. A GPU graphics processor power optimization scheduling method for tablet computers, characterized in that, include: Obtain the vertex throughput and triangle culling rate output by the vertex processing stage of the current frame of the tablet's graphics processor; The vertex throughput and the triangle culling rate are input into the frame load prediction model, and the predicted computational load of the rasterization and pixel coloring stages of the current frame is output. The frame load prediction model is a long short-term memory network, and the hidden layer state of the long short-term memory network is updated by the actual pixel coloring stage time of the previous frame. Based on the predicted load, a pre-built voltage and frequency lookup table is queried. Before the rasterization stage of the current frame starts, the configuration change of the graphics processor core voltage and frequency is completed. The weight matrices of the input gate, forget gate, and output gate of the long short-term memory network are trained and solidified offline during the tablet computer system initialization stage based on the vertex throughput, triangle culling rate, and actual pixel shading time of the historical rendering frames.

2. The method according to claim 1, characterized in that, Obtaining the vertex throughput and triangle culling rate output by the vertex processing stage of the current frame of the tablet computer's graphics processor includes: when the current frame enters the vertex shader execution stage, starting a hardware performance counter configured in the vertex processing pipeline of the graphics processor; the hardware performance counter accumulates the number of vertices passing through the vertex shader to obtain the vertex throughput, and accumulates the number of triangles culled by backface culling and view frustum clipping to obtain the total number of culled triangles. At the end of the vertex processing phase, the number of vertices and the total number of culled triangles in the hardware performance counter are read, and the ratio of the total number of culled triangles to the initial total number of triangles before entering the vertex processing phase is taken as the triangle culling rate.

3. The method according to claim 1, characterized in that, The hidden layer state of the Long Short-Term Memory network is updated by the actual pixel shading stage time of the previous frame, including: during the execution of the pixel shading stage of the previous frame, the actual pixel shading stage time is recorded by a periodic precision timer configured at the end of the pixel pipeline of the graphics processor. Before the vertex processing stage of the current frame begins, the difference between the actual pixel coloring stage time and the prediction computation load of the previous frame is used as an error feature vector. The error feature vector is concatenated with the hidden layer state vector output by the Long Short-Term Memory network in the previous frame. The concatenated vector is input into the forget gate of the Long Short-Term Memory network. The hidden layer state vector of the previous frame is multiplied element-wise by the output vector of the forget gate to obtain the updated hidden layer state of the current frame.

4. The method according to claim 1, characterized in that, The voltage frequency lookup table is pre-built according to the predicted load calculation, including: the voltage frequency lookup table uses the discrete calculated load level as an index to store the core clock frequency value and the power supply voltage value that uniquely correspond to the calculated load level; the voltage frequency lookup table also stores the voltage stabilization time corresponding to the core clock frequency value. The predicted calculation load is mapped to the closest calculation load level in the voltage frequency lookup table, and the corresponding core clock frequency value, power supply voltage value and voltage stabilization time are extracted. Calculate the idle time window between the end time of the vertex processing stage of the current frame and the start time of the rasterization stage. When the idle time window is greater than the voltage stabilization establishment time, trigger the instruction to send the power supply voltage value and the core clock frequency value.

5. The method according to claim 4, characterized in that, Before the rasterization phase of the current frame is started, the configuration change of the graphics processor core voltage and frequency is completed, including: when the idle time window is longer than the voltage stabilization time, sending a voltage adjustment command containing the supply voltage value to the power management integrated circuit of the tablet computer to drive the power management integrated circuit to adjust the output of the buck converter connected to the power rail of the graphics processor core; After detecting that the feedback voltage of the graphics processor core power rail has reached the supply voltage value, a frequency adjustment command containing the core clock frequency value is sent to the clock generator to control the clock generator to update the pixel shader clock signal output to the graphics processor. After the frequency of the clock signal is locked, a release signal is sent to the rasterization execution pipeline of the graphics processor to start the rasterization stage.

6. The method according to claim 1, characterized in that, The weight matrices of the input gate, forget gate, and output gate of the Long Short-Term Memory Network are trained and solidified offline during the initialization phase of the tablet computer system based on the vertex throughput, triangle culling rate, and actual pixel coloring time of historical rendering frames. This includes: during the initialization phase of the tablet computer system, traversing multiple preset benchmark 3D rendering test scenes, collecting the vertex throughput and triangle culling rate of each frame in the benchmark 3D rendering test scene as input samples, and collecting the actual pixel coloring time of the corresponding frame as label samples. Using mean squared error as the loss function, the backpropagation algorithm is used to iteratively train the initial model of the long short-term memory network, which includes the input gate, the forget gate and the output gate. The weight matrix corresponding to the iterative result that satisfies the convergence condition of the loss function is converted into a fixed-point number, and the fixed-point number is burned into the read-only memory of the microcontroller of the graphics processor.

7. The method according to claim 2, characterized in that, After reading the number of vertices and the total number of culled triangles in the hardware performance counter, the method further includes: comparing the number of vertices with a preset upper limit threshold for the number of vertices; when the number of vertices exceeds the upper limit threshold for the number of vertices, marking the vertex throughput as an abnormal state, and replacing the vertex throughput of the abnormal state with the vertex throughput of the previous frame. The triangle rejection rate is compared with a preset rejection rate lower limit threshold. When the triangle rejection rate is lower than the rejection rate lower limit threshold, the triangle rejection rate is marked as an abnormal state, and the triangle rejection rate in the abnormal state is replaced with the triangle rejection rate of the previous frame. The currently read vertex throughput and triangle culling rate are input into the frame load prediction model only when neither the vertex throughput nor the triangle culling rate is marked as abnormal.

8. The method according to claim 3, characterized in that, The difference between the actual pixel coloring stage time and the prediction calculation load of the previous frame is used as an error feature vector, including: obtaining the standard pixel processing cycle number corresponding to the prediction calculation load of the previous frame, and converting the actual pixel coloring stage time into the actual pixel processing cycle number. Calculate the absolute value of the difference between the actual number of pixel processing cycles and the standard number of pixel processing cycles; Extract the baseline load value corresponding to the computing load level of the predicted computing load in the previous frame, calculate the ratio of the absolute value of the difference to the baseline load value, and use the ratio as a dynamic scaling factor. The result of multiplying the absolute value of the difference by the dynamic scaling factor is taken as the scalar value of the error feature vector, and the scalar value is filled into the zero vector of a preset dimension to form the error feature vector.

9. The method according to claim 5, characterized in that, After calculating the idle time window between the end time of the vertex processing stage of the current frame and the start time of the rasterization stage, the method further includes: when the idle time window is less than or equal to the voltage stabilization establishment time, querying the voltage frequency lookup table for at least one low load level that is lower than the calculation load level corresponding to the predicted calculation load, and extracting the highest low load level among the at least one low load level whose voltage stabilization establishment time is less than the idle time window. Extract the degraded supply voltage value and degraded core clock frequency value corresponding to the highest and lowest load levels, send a voltage adjustment command containing the degraded supply voltage value to the power management integrated circuit, and after the feedback voltage of the graphics processor core power rail reaches the degraded supply voltage value, send a frequency adjustment command containing the degraded core clock frequency value to the clock generator, and after the frequency of the clock signal is locked, send the release signal to the rasterization execution pipeline.

10. The method according to claim 6, characterized in that, After the fixed-point numbers are burned into the read-only memory of the microcontroller of the graphics processor, the method further includes: during the vertex processing stage of the current frame, the microcontroller reads the fixed-point numbers of the weight matrices corresponding to the input gate, the forget gate and the output gate from the read-only memory; The vertex throughput and the triangle culling rate are quantized and truncated respectively, and converted into a set number of fixed-point inputs with a set bit width; The microcontroller performs matrix multiplication using the input fixed-point number and the fixed-point number of the weight matrix through shift and addition operations. The result of the matrix multiplication is dequantized and converted into a floating-point number through a preset lookup table. The converted floating-point number is used as the prediction calculation load output by the frame load prediction model.

Images