Rendering process-oriented frequency adjustment method and apparatus, and electronic device

Abstract

The embodiment of the invention discloses a rendering process-oriented frequency adjustment method and device and electronic equipment, and belongs to the technical field of frequency management of processors. Comprises: for a current stage in a rendering process of a current frame image, determining frame loss prediction information of the current stage, the frame loss prediction information being used for predicting whether frame loss will occur in the current frame image in the rendering process; determining a frequency adjustment strategy based on the frame loss prediction information, the frequency adjustment strategy being used for representing an adjustment mode of processing frequency; and adjusting the processing frequency of a target stage based on the frequency adjustment strategy, the target stage being a stage in the rendering process of the current stage or the next frame of image of the current frame of image.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of processor frequency management, and in particular to a frequency adjustment method, device, and electronic device for a rendering process. Background Art

[0002] With the advancement of hardware technology, current Android phones have more powerful processing capabilities, providing more resources for image rendering. However, due to the characteristics of mobile devices, power consumption is also a significant concern. To achieve a balance between performance and power consumption in the image rendering process, the scheduler often relies on allocating computing power and other resources. For example, the Windows-Assist Load Tracing (WALT) algorithm, a popular scheduler, measures the CPU load over a short period of time. Using multiple windows, it calculates CPU load and generates information for CPU frequency scaling and core migration. The WALT algorithm is essentially a window-based load prediction and does not make specific adjustments to the image rendering process. However, on current devices, to avoid user-perceived performance issues such as lag, the scheduler often allocates more performance, resulting in faster rendering. This trades power consumption for performance. For devices, it is more desirable to accurately utilize device resources to achieve a balance between performance and power consumption, thereby avoiding unnecessary power consumption. Summary of the Invention

[0003] The present invention provides a method, device, and electronic device for frequency adjustment in a rendering process. The technical solution is as follows:

[0004] In one aspect, an embodiment of the present application provides a frequency adjustment method for a rendering process, the method comprising:

[0005] For a current stage in a rendering process of a current frame image, determining frame loss prediction information of the current stage, wherein the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image;

[0006] Determining a frequency adjustment strategy based on the frame loss prediction information, wherein the frequency adjustment strategy is used to characterize an adjustment method of a processing frequency;

[0007] Based on the frequency adjustment strategy, the processing frequency of the target stage is adjusted, and the target stage is a stage in the rendering process of the current stage or the next frame image of the current frame image.

[0008] On the other hand, an embodiment of the present application provides a frequency adjustment device for a rendering process, the device comprising:

[0009] A first determining module is configured to determine, for a current stage in a rendering process of a current frame image, frame loss prediction information of the current stage, wherein the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image;

[0010] A second determining module is configured to determine a frequency adjustment strategy based on the frame loss prediction information, wherein the frequency adjustment strategy is used to characterize an adjustment method of a processing frequency;

[0011] An adjustment module is configured to adjust the processing frequency of a target stage based on the frequency adjustment strategy, where the target stage is a stage in a rendering process of the current stage or a next frame image of the current frame image.

[0012] On the other hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the frequency adjustment method for the rendering process as described in the above aspect.

[0013] On the other hand, an embodiment of the present application provides a chip, which includes a programmable logic circuit and / or program instructions. When the chip runs on an electronic device, the chip is used to implement the frequency adjustment method for the rendering process as described in the above aspects.

[0014] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the frequency adjustment method for the rendering process as described in the above aspect.

[0015] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the frequency adjustment method for the rendering process as described in the above aspects.

[0016] In an embodiment of the present application, the rendering process of the current frame image is divided into multiple stages. For the current stage, the processing frequency is adjusted based on the frame loss prediction information of the current stage. Therefore, by adjusting the processing frequency in stages, the processing frequency can be adjusted in a timely manner, which can improve the accuracy of the processing frequency adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram illustrating the division of rendering process stages according to an exemplary embodiment of the present application;

[0018] Figure 2 is a flow chart of a frequency adjustment method for a rendering process shown in an exemplary embodiment of the present application;

[0019] Figure 3 is a schematic diagram of frequency adjustment in the first stage shown in an exemplary embodiment of the present application;

[0020] Figure 4 is a schematic diagram showing a load difference between two adjacent frames of images according to an exemplary embodiment of the present application;

[0021] Figure 5 is a flowchart of a frequency adjustment method for a rendering process shown in another exemplary embodiment of the present application;

[0022] Figure 6 is a schematic diagram showing the execution start time and frame loss rate of a UI according to an exemplary embodiment of the present application;

[0023] Figure 7 is a schematic diagram of frequency adjustment in the second stage shown in an exemplary embodiment of the present application;

[0024] Figure 8 is a schematic diagram of frequency adjustment in the third stage according to an exemplary embodiment of the present application;

[0025] Figure 9 is a schematic diagram of frequency adjustment in the fourth stage according to an exemplary embodiment of the present application;

[0026] Figure 10 is a schematic diagram of first relationship data shown in an exemplary embodiment of the present application;

[0027] Figure 11 is a schematic diagram of second relationship data shown in an exemplary embodiment of the present application;

[0028] Figure 12 is a schematic diagram of target relationship data shown in an exemplary embodiment of the present application;

[0029] Figure 13 This is a structural block diagram of a frequency adjustment device for a rendering process shown in an exemplary embodiment of the present application;

[0030] Figure 14 It is a structural block diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0032] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0033] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the images involved in this application (previous frame image, current frame image, and next frame image) are all obtained with full authorization.

[0034] In the related art, in order to achieve a balance between the performance and power consumption of an electronic device, the CPU load value of the electronic device in a historical time period is determined, and a processing frequency is allocated for image rendering based on the CPU load value. Then, the electronic device processes the image frame based on the allocated processing frequency. For example, when the CPU load value is large, a higher processing frequency is allocated for image rendering; when the CPU load value is small, a lower processing frequency is allocated for image rendering. This allocation of processing frequency based on load tracking logic is universal. At this time, if there is an irrelevant load running in the background, the CPU processing frequency will be too high. That is, the related art does not perform on-demand frequency adjustment according to the processing stage of the image rendering process.

[0035] The rendering frequency of the electronic device can be 30 frames per second (fps), 60fps, 90fps or 120fps, etc.; in the image rendering process of the electronic device, taking the rendering frequency of the electronic device as 60fps as an example, the expected rendering time of any frame image should be controlled within 16.67 milliseconds; when the actual rendering time of a frame image exceeds 16.67 milliseconds, the frame image belongs to a timeout frame, that is, frame loss may occur for the user. Frame loss will make the user feel that the picture is stuck, and the stuck situation will seriously affect the user experience; and the more complex the image rendering, the more likely it is to cause stuck. It can be seen that the method for determining the processing frequency in the related art cannot well control the overall rhythm of the image rendering process.

[0036] In an embodiment of the present application, the processing frequency is adjusted in real time using the frame process in the rendering process of the current frame image, so that the processing frequency can be adjusted more accurately while ensuring that the rendering process is completed on time. In order to achieve accurate regulation of the frame image rendering process, based on the load tracking of related technologies, the rendering process is regulated in stages. The rendering of a frame image is mainly completed by the CPU and the image processor (Graphics Processing Unit, GPU); the CPU mainly completes input processing, animation processing, measurement, layout, drawing and synchronization operations by the UI thread. Input processing refers to the UI thread processing user input events such as touch and key presses (INPUT events), and passing these events to the corresponding view or control for processing through the INPUT event distribution mechanism. Animation processing refers to the UI thread being responsible for drawing and updating animations to ensure that the animation effect is smooth and synchronized with user interaction. Measurement refers to the electronic device determining the size of each child view according to the constraints passed by the parent view during the measurement phase. Each view will call the onMeasure method to calculate its own size based on the incoming width and height measurement information. Layout refers to the electronic device performing layout after the measurement is completed, calling the onLayout method, and setting the position of each view according to the measurement results. Drawing (Draw) means that after the layout is completed, the electronic device performs a drawing operation and calls the onDraw method to perform specific drawing operations.

[0037] After obtaining information about UI elements and layout, the UI thread passes it to the Render thread. The Render thread processes the UI elements and layout, and when the Render thread is finished, it submits the drawing task to the GPU hardware module for rendering. After the GPU hardware module completes the rendering, SurfaceFlinger submits the rendering buffer to the hardware compositor for compositing, and finally transmits it to the display for display.

[0038] refer to Figure 1Based on the UI thread and the Render thread, the electronic device divides the rendering process of a frame image into four stages, namely the first stage, the second stage, the third stage and the fourth stage. The first stage is the waiting stage before the UI thread executes after receiving the vertical synchronization (Vertical Synchronization, VSYNC) signal, and the start time of the first stage is when the VSYNC signal of the current frame image is received, and the end time of the first stage is when the UI thread starts to execute. The VSYNC signal is used to instruct the rendering of the current frame image; for example, before rendering the current frame image, the UI thread can be in a dormant state; after receiving the VSYNC signal, the electronic device wakes up the UI thread and then instructs the UI thread to render the current frame image.

[0039] The second stage is the UI thread execution stage, and the start time of the second stage is when the UI thread starts executing, and the end time of the second stage is when the UI thread finishes executing. The third stage is the Render thread execution stage. At the end of the third stage, the Render thread is submitted to the GPU hardware module for execution, and the start time of the third stage is when the UI thread finishes executing, and the end time of the third stage is when the Render thread finishes executing. The fourth stage is the stage from when the rendering thread completes rendering to when the VSYNC signal of the next frame of image is received, that is, the remaining time stage after the rendering work of the current frame of image is completed, and the start time of the fourth stage is when the Render thread finishes executing, and the end time of the fourth stage is when the VSYNC signal of the next frame of image is received.

[0040] In an embodiment of the present application, the electronic device divides the image rendering process into four stages, predicts the frame loss situation at each stage, and optimizes the load tracking mechanism based on the predicted frame loss situation to adjust the processing frequency at each stage. Thus, through the staged processing frequency adjustment, timely adjustment of the processing frequency is achieved, thereby improving the accuracy of the processing frequency adjustment. For example, for a certain stage of the rendering process, if frame loss may occur at that stage, the processing frequency of that stage is increased, thereby reducing the occurrence of frame loss and thus reducing the phenomenon of jamming; if frame loss is unlikely to occur at that stage, the processing frequency of that stage is reduced, thereby reducing power consumption without sacrificing performance.

[0041] refer to Figure 2 , which shows a flow chart of a frequency adjustment method for a rendering process provided by an exemplary embodiment of the present application. The method may include the following steps:

[0042] Step 201: For a current stage in a rendering process of a current frame image, the electronic device determines frame loss prediction information of the current stage, where the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image.

[0043] Prior to this step, the electronic device divides the rendering process of the current frame image into multiple stages: the first stage, the second stage, the third stage, and the fourth stage. The first stage is the waiting stage before the UI thread executes after receiving the VSYNC signal. The second stage is the UI thread execution stage. The third stage is the Render thread execution stage. At the end of the third stage, the Render thread is submitted to the GPU hardware module for execution. The fourth stage is the stage from the completion of the Render thread rendering to the receipt of the VSYNC signal for the next frame image, that is, the remaining time stage after the rendering work of the current frame image is completed. The current stage can be any of the multiple stages.

[0044] Step 202: The electronic device determines a frequency adjustment strategy based on the frame loss prediction information. The frequency adjustment strategy is used to represent a method for adjusting the processing frequency.

[0045] For example, when the current stage is the second stage in the rendering process, the processing frequency may be the processing frequency of the CPU; when the current stage is the third stage in the rendering process, the processing frequency may be the processing frequency of the CPU.

[0046] When the frame loss prediction information is used to predict that the current frame image will experience frame loss during the rendering process, the electronic device determines a frequency adjustment strategy to represent an increase in the processing frequency; when the frame loss prediction information is used to predict that the current frame image will not experience frame loss during the rendering process, the electronic device determines a frequency adjustment strategy to represent a decrease in the processing frequency.

[0047] Step 203: The electronic device adjusts the processing frequency of the target stage based on the frequency adjustment strategy. The target stage is the stage in the rendering process of the current stage or the next frame image of the current frame image.

[0048] In one possible implementation, when the frequency adjustment strategy is used to represent an increase in processing frequency, the electronic device increases the processing frequency of the target stage, thereby reducing the occurrence of frame loss and further reducing the lag phenomenon; when the frequency adjustment strategy is used to represent a reduction in processing frequency, the electronic device reduces the processing frequency of the target stage, thereby reducing the power consumption of the electronic device.

[0049] For example, when the target stage is the second stage, the step of the electronic device increasing the processing frequency of the target stage may be: the electronic device increases the processing frequency of the CPU in the target stage; when the target stage is the third stage, the step of the electronic device increasing the processing frequency of the target stage may be: the electronic device increases the processing frequency of the CPU in the target stage.

[0050] For example, when the target stage is the second stage, the step of the electronic device reducing the processing frequency of the target stage may be: the electronic device reduces the processing frequency of the CPU in the target stage; when the target stage is the third stage, the step of the electronic device reducing the processing frequency of the target stage may be: the electronic device reduces the processing frequency of the CPU in the target stage.

[0051] It should be noted that the electronic device may also determine the end time of the Render thread of the third stage. When the end time of the Render thread of the third stage is later than the third preset time, the electronic device may increase the processing frequency of the GPU.

[0052] In an embodiment of the present application, the rendering process of the current frame image is divided into multiple stages. For the current stage, the processing frequency is adjusted based on the frame loss prediction information of the current stage. Therefore, through the stage-by-stage processing frequency adjustment, the processing frequency can be adjusted in a timely manner, which can improve the accuracy of the processing frequency adjustment.

[0053] refer to Figure 3 , which shows a flow chart of a frequency adjustment method for a rendering process provided by an exemplary embodiment of the present application. The method may include the following steps:

[0054] Step 301: For a current stage in a rendering process of a current frame image, the electronic device determines frame loss prediction information of the current stage, where the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image.

[0055] The first case: the current stage is the first stage in the rendering process; the first stage is the waiting stage before the UI thread executes after receiving the VSYNC signal; when the VSYNC signal is received, it is the beginning of the rendering process of the current frame image; how to adjust the processing frequency of the rendering process at this time mainly depends on the locality between frames; for example, when the previous frame image is a high-load frame, the current frame image is likely to be a high-load frame; when the previous frame image is a low-load frame, the current frame image is likely to be a low-load frame. Therefore, in the embodiment of the present application, the adjustment of the processing frequency of the first stage is based on the locality between frames; for example, refer to Figure 4 , the electronic device counts the difference in load between two adjacent frames of images, based on Figure 4It can be seen that the difference in load between two adjacent frames of image is mostly close to 0; therefore, when the previous frame of image is completed too early, the processing frequency can be reduced in the first stage of the current frame of image, thereby ensuring that the current frame of image is rendered on time and saving power consumption of the electronic device; when the previous frame of image is completed late, the processing frequency can be increased in the first stage of the current frame of image, thereby ensuring that the current frame of image can be rendered on time.

[0056] Accordingly, the step of the electronic device determining the frame loss prediction information of the current stage may be:

[0057] The electronic device determines a frequency adjustment flag of a previous frame image of a current frame image; when the frequency adjustment flag is a reduction flag, the frame loss prediction information is determined to be first prediction information, and the first prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process; when the frequency adjustment flag is an increase flag, the frame loss prediction information is determined to be second prediction information, and the second prediction information is used to predict that frame loss will occur in the rendering process of the current frame image.

[0058] The decrease flag can be a frequency reduction flag; the increase flag can be a frequency increase flag. Figure 5 , taking the frequency adjustment flag as a reduction flag as an example, when the VSYNC signal is received, it is determined whether the frequency adjustment flag of the fourth stage of the previous frame image is an increase flag or a decrease flag; when the previous frame image is rendered quickly, the frequency adjustment flag is a decrease flag; due to the existence of inter-frame load locality; the electronic device can adjust the processing frequency of the first stage of the current frame image based on the actual rendering time of the previous frame image.

[0059] Correspondingly, the steps for the electronic device to determine the frequency adjustment flag of the previous frame image of the current frame image can be: the electronic device determines the actual rendering time of the previous frame image; when the actual rendering time of the previous frame image is less than the first preset time, the frequency adjustment flag of the previous frame image is determined to be a reduction flag; when the actual rendering time of the previous frame image is greater than the first preset time, the frequency adjustment flag of the previous frame image is determined to be an increase flag.

[0060] For example, continue to refer to Figure 5 , the electronic device determines the rendering time of each frame image; taking 80% as an example, the rendering time of each frame image is controlled to be around 80% of the expected rendering time; therefore, when the actual rendering time of the previous frame image is less than 80% of the expected rendering time, the frequency adjustment flag of the previous frame image is determined to be a decrease flag; when the actual rendering time of the previous frame image is greater than 80% of the expected rendering time, the frequency adjustment flag of the previous frame image is determined to be an increase flag.

[0061] Accordingly, the step of the electronic device determining the first preset duration may be: the electronic device determines the expected rendering duration of the previous frame image; based on the expected rendering duration of the previous frame image and the first preset coefficient, the first preset duration is determined, and the first preset duration is less than the expected rendering duration of the previous frame image. For example, the electronic device determines the product of the expected rendering duration of the previous frame image and the first preset coefficient to obtain the first preset duration. The first preset coefficient can be set and changed as needed; in the embodiment of the present application, the first preset coefficient is not specifically limited; for example, the first preset coefficient can be 75%, 80% or 85%.

[0062] The second situation: the current stage is the second stage in the rendering process, and the second stage is the UI thread execution stage; the UI thread is the core thread in the rendering process. In a high-load scenario, the CPU where the UI thread runs is busy, and it is easy for the CPU to be occupied by other threads, resulting in a late start time for the UI thread. Therefore, in the embodiment of the present application, the electronic device analyzes the correlation between the execution start time of the UI thread and the frame loss; refer to Figure 6 For example, if the rendering frequency of the electronic device is 60fps, the frame period is 16.67 milliseconds; Figure 6 In the example, the electronic device counts the distribution of the execution start time of the UI thread of the currently running application based on 1 millisecond. Figure 6 It can be seen that the execution start time of the UI thread of most frame images is within 2 milliseconds after receiving the VSYNC signal, and the frame loss rate is low at this time; when the execution start time of the UI thread of the frame image is 2 seconds after receiving the VSYNC signal, the frame loss rate shows an overall increasing trend, and the later the execution start time of the UI thread, the higher the risk of frame loss. Therefore, it is necessary for electronic devices to adjust the processing frequency based on the execution start time of the UI thread; when the electronic device detects that the execution start time of the UI thread is late, the electronic device can increase the processing frequency to increase the processing speed of the UI thread and reduce the risk of frame loss; when the electronic device detects that the execution start time of the UI thread is early, the electronic device reduces the processing frequency, thereby reducing the power consumption of the electronic device.

[0063] Accordingly, the step of the electronic device determining the frame loss prediction information of the current stage may be:

[0064] The electronic device determines the execution delay of the UI thread, where the execution delay is the time difference between the execution start time of the UI thread and the reception time of the VSYNC signal; when the execution delay is less than the second preset duration, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the execution delay is greater than the third preset duration, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process, and the third preset duration is greater than the second preset duration.

[0065] If the execution delay of the UI thread is greater than the second preset duration and less than the third preset duration, the electronic device determines not to adjust the processing frequency. The second preset duration and the third preset duration can be set and changed as needed; in the embodiment of the present application, the second preset duration and the third preset duration are not specifically limited; for example, the third preset duration can be 3 milliseconds or 4 milliseconds; the second preset duration can be 1 millisecond or 2 milliseconds.

[0066] The step of the electronic device determining the execution delay of the UI thread may be: the electronic device determines the difference between the execution start time of the UI thread and the time when the VSYNC signal of the current frame image is received to obtain the execution delay of the UI thread.

[0067] In another possible implementation, for example, referring to Figure 7 , the electronic device sets two time critical values, namely a first preset time and a second preset time; based on the relationship between the execution start time of the UI thread and the first preset time and the second preset time, the frame loss prediction information of the current stage is determined; accordingly, the electronic device determines the execution start time of the UI thread; when the execution start time of the UI thread is earlier than the first preset time, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the execution start time of the UI thread is later than the second preset time, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process; the second preset time is later than the first preset time.

[0068] If the execution start time of the UI thread is earlier than the first preset time, it indicates that the execution start time of the UI thread is early, indicating that there is a possibility of reducing the processing frequency; therefore, the electronic device determines the frame loss prediction information as the first prediction information, and the first prediction information is used to cause the electronic device to reduce the processing frequency. If the execution start time of the UI thread is later than the second preset time, it indicates that the execution start time of the UI thread is late, indicating that there is a risk of frame loss; therefore, the electronic device determines the frame loss prediction information as the second prediction information, and the second prediction information is used to cause the electronic device to increase the processing frequency. If the execution start time of the UI thread is between the first preset time and the second preset time, the processing frequency is not adjusted, and thus load tracking is not intervened.

[0069] The third situation: the current stage is the third stage in the rendering process, and the third stage is the Render thread execution stage. In the third stage, the work of the UI thread has been completed, and the Render thread is now the core thread of the third stage; if the UI thread of the previous frame image is completed later, then the risk of frame loss in the current frame image will increase. In order to speed up the processing speed of the current frame image, it is necessary to increase the processing frequency of the third stage. The processing frequency of the third stage can be the processing frequency of the CPU. It should be noted that the electronic device can also determine the end time of the Render thread of the third stage. When the end time of the Render thread of the third stage is later than the third preset time, the electronic device can increase the processing frequency of the GPU.

[0070] Accordingly, the step of the electronic device determining the frame loss prediction information of the current stage may be:

[0071] The electronic device determines the thread completion time of the previous stage of the third stage; when the thread completion time is less than the fourth preset time, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the thread completion time is greater than the fourth preset time, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process.

[0072] The completion time of the thread in the previous stage is the completion time of the UI thread in the second stage. The process of determining the fourth preset time can be: the electronic device determines the expected rendering time of the current frame image; based on the expected rendering time of the current frame image and the second preset coefficient, the fourth preset time is determined, and the fourth preset time is less than the expected rendering time of the current frame image; for example, the product of the expected rendering time of the current frame image and the second preset coefficient is determined to obtain the fourth preset time. The second preset coefficient can be set and changed as needed. In the embodiment of the present application, the second preset coefficient is not specifically limited; for example, the second preset coefficient can be 75%, 80% or 85%.

[0073] For example, reference Figure 8 , taking the electronic device reducing the processing frequency as an example for explanation; the electronic device determines whether the thread completion time of the UI thread of the current frame is less than 80% of the expected rendering time of the current frame image; when the thread completion time of the UI thread of the current frame is less than 80% of the expected rendering time of the current frame image, it is determined that the frame loss prediction information is the first prediction information, and the first prediction information enables the electronic device to reduce the processing frequency.

[0074] The fourth situation: the current stage is the fourth stage in the rendering process. The fourth stage is the stage from the completion of the rendering of the Render thread to the reception of the VSYNC signal of the next frame image. The end of the execution of the Render thread means that the rendering work of the current frame image has been completed. At this time, if it exceeds the frame period (for example, if the rendering frequency of the electronic device is 60fps, the frame period is 16.67 milliseconds), it means that the current frame image will be lost. In order to achieve the regulation of the fourth stage, the electronic device sets 80% of the frame period as the fifth preset duration (critical duration). When the actual rendering duration of the current frame image is greater than the fifth preset duration, it means that the rendering load at this time is large and the processing of rendering-related work cannot be responded to in time. Therefore, in the rendering process of the next frame image, the processing frequency needs to be increased. Since the rendering work of the current frame image has been completed, the operation of increasing the processing frequency is not performed for the current frame image. When the actual rendering duration of the current frame image is less than the fifth preset duration, it means that the rendering load at this time is small and the idle time in the rendering process of the current frame image is large. Therefore, in the rendering process of the next frame image, the processing frequency needs to be reduced.

[0075] Accordingly, the step of the electronic device determining the frame loss prediction information of the current stage may be:

[0076] The electronic device determines the actual rendering time of the current frame image; when the actual rendering time of the current frame image is less than the fifth preset time, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the actual rendering time of the current frame image is greater than the fifth preset time, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process.

[0077] For example, reference Figure 9 The actual rendering completion time of the current frame image is the execution end time of the Render thread; and the actual rendering duration of the current frame image is the time difference between the execution end time of the Render thread and the time when the VSYNC signal is received.

[0078] The fifth preset duration may be determined by: the electronic device determining the expected rendering duration of the current frame image; determining the fifth preset duration based on the expected rendering duration of the current frame image and the third preset coefficient, wherein the fifth preset duration is less than the expected rendering duration of the current frame image; for example, determining the product of the expected rendering duration of the current frame image and the third preset coefficient to obtain the fifth preset duration. The third preset coefficient can be set and changed as needed. In the embodiment of the present application, the third preset coefficient is not specifically limited; for example, the third preset coefficient may be 75%, 80%, or 85%.

[0079] One thing that needs to be explained is that the first preset coefficient, the second preset coefficient and the third preset coefficient can be the same or different; in the embodiment of the present application, the first preset coefficient, the second preset coefficient and the third preset coefficient are all the same and are 80% as an example for explanation.

[0080] In the embodiment of the present application, the electronic device analyzes the locality and correlation of multiple stages, analyzes the relationship between the adjustment of multiple stages and frame loss, fully utilizes the characteristics of the inter-frame load, sets the appropriate trigger timing, and adjusts the processing frequency. In addition, by dividing the rendering process of the current frame image into multiple stages, the embodiment of the present application is equivalent to making the load prediction based on the frame image more precise. Through the staged processing frequency adjustment, the processing frequency can be adjusted in a timely manner, thereby better cooperating with the scheduler's scheduling of the processing frequency.

[0081] Step 302: The electronic device determines a frequency adjustment strategy based on the frame loss prediction information. The frequency adjustment strategy is used to represent a method for adjusting the processing frequency.

[0082] In the case where the frame loss prediction information is used to predict that the current frame image will experience frame loss in the rendering process, the electronic device determines that the frequency adjustment strategy is used to characterize an increase in the processing frequency, that is, in the case where the frame loss prediction information is the second prediction information, the electronic device determines that the frequency adjustment strategy is used to characterize an increase in the processing frequency; in the case where the frame loss prediction information is used to predict that the current frame image will not experience frame loss in the rendering process, the electronic device determines that the frequency adjustment strategy is used to characterize a reduction in the processing frequency, that is, in the case where the frame loss prediction information is the first prediction information, the electronic device determines that the frequency adjustment strategy is used to characterize a reduction in the processing frequency.

[0083] In one possible implementation, when the frequency adjustment strategy is used to represent an increase in processing frequency, the electronic device increases the processing frequency of the target stage, thereby reducing the occurrence of frame loss and further reducing the lag phenomenon; when the frequency adjustment strategy is used to represent a reduction in processing frequency, the electronic device reduces the processing frequency of the target stage, thereby reducing the power consumption of the electronic device.

[0084] For example, when the target stage is the second stage, the step of the electronic device increasing the processing frequency of the target stage may be: the electronic device increases the processing frequency of the CPU in the target stage; when the target stage is the third stage, the step of the electronic device increasing the processing frequency of the target stage may be: the electronic device increases the processing frequency of the CPU in the target stage.

[0085] For example, when the target stage is the second stage, the step of the electronic device reducing the processing frequency of the target stage may be: the electronic device reduces the processing frequency of the CPU in the target stage; when the target stage is the third stage, the step of the electronic device reducing the processing frequency of the target stage may be: the electronic device reduces the processing frequency of the CPU in the target stage.

[0086] It should be noted that the electronic device may also determine the end time of the Render thread of the third stage. When the end time of the Render thread of the third stage is later than the third preset time, the electronic device may increase the processing frequency of the GPU.

[0087] In an embodiment of the present application, the rendering process of the current frame image is divided with the UI thread and the Render thread as entry points, and the rendering process of the current frame image is dynamically managed by combining the execution stages of the UI thread and the Render thread; in the case of performance loss, the performance is remedied by increasing the processing frequency; when the rendering speed is too fast, the power consumption of the electronic device is reduced by reducing the processing frequency.

[0088] Step 303: The electronic device determines an adjustment coefficient for a target phase based on the current phase and the frequency adjustment strategy.

[0089] The adjustment coefficient is used to adjust the processing frequency of the target stage; for example, the adjustment coefficient may be an adjustment coefficient for increasing the processing frequency of the target stage; or, the adjustment coefficient may be an adjustment coefficient for decreasing the processing frequency of the target stage.

[0090] In one possible implementation, the electronic device adjusts the processing frequency of the first stage based on the frame loss prediction information of the first stage; accordingly, the current stage is the first stage in the rendering process, and the target stage is the first stage in the rendering process. In another possible implementation, the electronic device adjusts the processing frequency of the second stage based on the frame loss prediction information of the second stage; accordingly, the current stage is the second stage in the rendering process, and the target stage is the second stage in the rendering process. In another possible implementation, the electronic device adjusts the processing frequency of the third stage based on the frame loss prediction information of the third stage; accordingly, the current stage is the third stage in the rendering process, and the target stage is the third stage in the rendering process. In another possible implementation, the electronic device adjusts the processing frequency of the first stage of the next frame image based on the frame loss prediction information of the fourth stage of the current frame image; accordingly, the current stage is the fourth stage in the rendering process, and the target stage is the first stage of the next frame image.

[0091] The first case: the current stage is the first stage in the rendering process, the target stage is the first stage in the rendering process, and the first stage is the waiting stage before the UI thread executes after receiving the VSYNC signal; that is, the electronic device determines the adjustment coefficient of the first stage based on the frame loss prediction information of the first stage.

[0092] Correspondingly, the electronic device may determine the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy as follows: the electronic device may determine the first adjustment coefficient of the first stage based on the actual rendering completion time of the previous frame image of the current frame image and the frequency adjustment strategy.

[0093] In one possible implementation, the electronic device may determine the first adjustment coefficient of the first stage based on the actual rendering completion time and frequency adjustment strategy of the previous frame image of the current frame image. The steps may be as follows: the electronic device determines the actual rendering duration of the previous frame image based on the actual rendering completion time of the previous frame image and the reception time of the VSYNC signal of the previous frame image; determines the actual duration coefficient of the previous frame image based on the actual rendering duration and the expected rendering duration of the previous frame image; and determines the first adjustment coefficient of the first stage based on the actual duration coefficient of the previous frame image and the frequency adjustment strategy.

[0094] For example, the electronic device may determine the actual rendering duration of the previous frame image based on the actual rendering completion time of the previous frame image and the reception time of the VSYNC signal of the previous frame image as follows: the electronic device determines the time difference between the actual rendering completion time of the previous frame image and the reception time of the VSYNC signal of the previous frame image to obtain the actual rendering duration of the previous frame image.

[0095] For example, the electronic device may determine the actual duration coefficient of the previous frame image based on the actual rendering duration and the expected rendering duration of the previous frame image as follows: the electronic device determines the ratio between the actual rendering duration and the expected rendering duration of the previous frame image to obtain the actual duration coefficient of the previous frame image.

[0096] In order to control the CPU processing frequency from being reduced too low or increased too high, the electronic device sets an upper limit coefficient and a lower limit coefficient to constrain the first adjustment coefficient, thereby avoiding the risk of frame loss caused by the CPU processing frequency being too low, or excessive power consumption caused by the CPU processing frequency being too high. Accordingly, the electronic device can determine the first adjustment coefficient of the first stage based on the actual duration coefficient of the previous frame image and the frequency adjustment strategy by:

[0097] When the frequency adjustment strategy is used to represent an increase in the processing frequency, the electronic device determines the first adjustment coefficient of the first stage based on the actual duration coefficient of the previous frame image and the first preset duration coefficient, and the first adjustment coefficient is the minimum duration coefficient of the actual duration coefficient of the previous frame image and the first preset duration coefficient; when the frequency adjustment strategy is used to represent a reduction in the processing frequency, the first adjustment coefficient of the first stage is determined based on the actual duration coefficient of the previous frame image and the second preset duration coefficient, and the first adjustment coefficient is the maximum duration coefficient of the actual duration coefficient of the previous frame image and the second preset duration coefficient, and the first preset duration coefficient is greater than the second preset duration coefficient.

[0098] The first preset duration coefficient is the upper limit coefficient; the second preset duration coefficient is the lower limit coefficient; and, both the first preset duration coefficient and the second preset duration coefficient can be set and changed as needed; in the embodiment of the present application, there is no specific limitation on the first preset duration coefficient and the second preset duration coefficient; for example, the first preset duration coefficient can be 1.2 or 1.25; the second preset duration coefficient can be 0.75 or 0.8, etc.

[0099] For example, continue to refer to Figure 5 , taking reducing the processing frequency as an example, the first adjustment coefficient factor = max{(render_end-vsync / frame_time, 0.75)}; wherein render_end represents the rendering end time of the previous frame image, vsync represents the receiving time of the VSYNC signal of the previous frame image, and frame_time represents the expected rendering duration; when the rendering frequency of the electronic device is 60fps, the expected rendering duration of the previous frame image is 16.67 milliseconds.

[0100] The second case: the current stage is the second stage in the rendering process, the target stage is the second stage, and the second stage is the UI thread execution stage; that is, the electronic device determines the adjustment coefficient of the second stage based on the frame loss prediction information of the second stage.

[0101] Correspondingly, the electronic device determines the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy as follows: when the frequency adjustment strategy is used to characterize an increase in the processing frequency, the electronic device determines the second adjustment coefficient of the second stage; when the frequency adjustment strategy is used to characterize a decrease in the processing frequency, the electronic device determines the third adjustment coefficient of the second stage, the second adjustment coefficient and the third adjustment coefficient satisfy the target relationship data, and the target relationship data is used to characterize the relationship between power consumption and performance, that is, the target relationship data is used to balance the relationship between power consumption and performance of the electronic device, so that the performance requirements of the electronic device can be met while reducing the power consumption of the electronic device.

[0102] The process of an electronic device determining target relationship data can be: the electronic device determines first relationship data based on historical rendering data, and the first relationship data is used to characterize the power consumption relationship satisfied by the second adjustment coefficient and the third adjustment coefficient; based on the historical rendering data, the electronic device determines second relationship data, and the second relationship data is used to characterize the performance relationship satisfied by the second adjustment coefficient and the third adjustment coefficient; based on the first relationship data and the second relationship data, the target relationship data is determined.

[0103] For example, the electronic device may determine the target relationship data based on the first relationship data and the second relationship data in the following steps: the electronic device determines the union of the second adjustment coefficient and the third adjustment coefficient constrained by the first relationship data and the second adjustment coefficient and the third adjustment coefficient constrained by the second relationship data, respectively, to obtain the target relationship data, that is, the second adjustment coefficient constrained by the target relationship data satisfies both the first relationship data and the second relationship data, and similarly, the second adjustment coefficient constrained by the target relationship data satisfies both the first relationship data and the second relationship data.

[0104] The process of determining the first relational data:

[0105] The power model of electronic equipment is P = CV 2 f≈f 3 Where P represents the power of the electronic device, C represents the load capacitance, V represents the power supply voltage, and f represents the rendering frequency. The power consumption model of the electronic device is: E = Pt = CV 2 ft≈f 3t; where E represents the power consumption of the electronic device and t represents the rendering time. The performance model of the electronic device is t≈1 / f. Since the rendering process includes multiple stages, some of which may increase the processing frequency, some may decrease the processing frequency, and some may keep the processing frequency unchanged; therefore, when determining the power consumption of the electronic device, it is necessary to consider the probability of adjusting the processing frequency; that is, the power consumption model of the electronic device will be updated to: E=Pt*P r Among them, P r The probability of adjusting the processing frequency (frequency modulation probability) is represented by the probability of increasing the processing frequency, decreasing the processing frequency, and maintaining the processing frequency unchanged. The electronic device accumulates the corresponding power consumption to obtain the mathematical expectation of power consumption.

[0106] For example, taking increasing the processing frequency as an example, a represents the second adjustment coefficient for increasing the processing frequency, and the power consumption of the electronic device during the period of increasing the processing frequency is: E up =Pt(time to increase processing frequency)*P r (increase the probability of processing frequency); where E up Indicates the power consumption of the electronic device during the period of increasing the processing frequency. t (duration of increasing the processing frequency) is used to characterize the mean duration of increasing the processing frequency, and t (duration of increasing the processing frequency) is the time calculated after adjusting the processing frequency based on the method provided in the embodiment of the present application, which requires the electronic device to first determine the second adjustment coefficient, and then determine the power consumption of the electronic device based on the second adjustment coefficient; however, the purpose of establishing the power consumption model in the embodiment of the present application is to determine the second adjustment coefficient, which leads to mutual dependence between the data. Therefore, the electronic device collects and counts the duration t (the first duration of increasing the processing frequency) of each time period without setting the second adjustment coefficient; then converts t (duration of increasing the processing frequency) = t (the first duration of increasing the processing frequency) / a, where a represents the second adjustment coefficient. Accordingly, the power consumption of the electronic device during the period of increasing the processing frequency can be updated as:

[0107] Formula 1: E up =(af) 3 *t(the first duration of increasing processing frequency) / a*P r (First probability of increasing processing frequency) = a 2 f 3 *t(the first duration of increasing processing frequency)*P r (Increase the first probability of processing frequency).

[0108] Similarly, based on the above principle, the power consumption of electronic devices during the period of reducing processing frequency can be obtained as formula 2: E down =b 2 f 3*t(second duration to reduce processing frequency)*P r (Second probability of reducing processing frequency).

[0109] Similarly, based on the above principle, the power consumption of electronic devices during the period of maintaining the same processing frequency can be obtained as formula 3: E de =f 3 *t(the third duration to keep the processing frequency unchanged)*P r (The third probability keeping the treatment frequency constant).

[0110] The electronic device can determine the power consumption of the electronic device after adjusting the processing frequency based on Formula 1, Formula 2, and Formula 3; and the power consumption before the device adjusts the processing frequency (which can be called the control group power consumption) can be determined based on the default probability of 100%, and t is the total duration of the sum of the above three durations; accordingly, the power consumption of the electronic device before adjusting the processing frequency is Formula 4: E cg =f 3 *t(total duration)*100%.

[0111] The power consumption benefit of the electronic device is determined based on Formula 1, Formula 2, Formula 3 and Formula 4 as Formula 5: E′=E cg -(E up +E down +E de ); accordingly, the historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process; the electronic device can determine the first relationship data based on the historical rendering data by the following steps (1)-(5), including:

[0112] (1) The electronic device determines first power consumption relationship data based on the first duration and the first probability, where the first power consumption relationship data is used to represent the relationship between the second adjustment coefficient and the first power consumption consumed during the first duration.

[0113] Based on the power consumption model of increasing the processing frequency corresponding to Formula 1, the electronic device substitutes the first duration and the first probability into the power consumption model to obtain first power consumption relationship data. For example, Formula 1: E up =a 2 f 3 *t(the first duration of increasing processing frequency)*P r (the first probability of increasing the processing frequency), the electronic device combines t (the first duration of increasing the processing frequency) and P r Substituting (the first probability of increasing the processing frequency) into Formula 1 yields the relationship data that is the first power consumption relationship data.

[0114] In one possible implementation, the first duration includes sub-durations corresponding to multiple stages, and the first probability includes sub-probabilities corresponding to multiple stages; accordingly, the step for the electronic device to determine the first power consumption relationship data based on the first duration and the first probability may be: for any stage, the electronic device determines the first sub-power consumption relationship data corresponding to the stage based on the sub-duration corresponding to the stage and the sub-probability corresponding to the stage; the first sub-power consumption relationship data corresponding to multiple stages are accumulated to obtain the first power consumption relationship data.

[0115] For example, historical rendering data of the current application running on the electronic device is shown in Table 1 below:

[0116] Table 1

[0117]

[0118] The electronic device determines E based on formula 1 and the data in Table 1. up0 =2.7453*6.38%a 2 f 3 =0.1752a 2 f 3 ;E up1 =0.0891a 2 f 3 ;E up2 =0.0779a 2 f 3 ;E up3 =0, then the first relation data can be E up0 +E up1 +E up2 +E up3 =0.2642a 2 f 3 .

[0119] (2) The electronic device determines second power consumption relationship data based on the second duration and the second probability, where the second power consumption relationship data is used to represent the relationship between the third adjustment coefficient and the second power consumption consumed during the second duration.

[0120] Based on the power consumption model for reducing the processing frequency corresponding to Formula 2, the electronic device substitutes the second duration and the second probability into the power consumption model to obtain second power consumption relationship data. For example, Formula 2: E down =b 2 f 3 *t(second duration to reduce processing frequency)*P r (the second probability of reducing the processing frequency). The electronic device combines t (the second time length of reducing the processing frequency) and P r Substituting the second probability of reducing the processing frequency into formula 2 yields the relationship data that is the second power consumption relationship data.

[0121] In one possible implementation, the second duration includes sub-durations corresponding to multiple stages, and the second probability includes sub-probabilities corresponding to multiple stages; accordingly, the step for the electronic device to determine the second power consumption relationship data based on the second duration and the second probability may be: for any stage, the electronic device determines the second sub-power consumption relationship data corresponding to the stage based on the sub-duration corresponding to the stage and the sub-probability corresponding to the stage; and accumulates the second sub-power consumption relationship data corresponding to multiple stages to obtain the second power consumption relationship data.

[0122] The electronic device determines E based on formula 2 and the data in Table 1. down0 =1.5682*93.62%b 2 f 3 =1.4681b 2 f 3 ;E down1 =1.8834b 2 f 3 ;E down2 =3.7895b 2 f 3 ;E down3 =8.9698 2 f 3 , then the second relational data can be E dowm0 +E down1 +E dowm2 +E down3 =16.1108b 2 f 3 .

[0123] (3) The electronic device determines a third power consumption based on the third time period and the third probability, where the third power consumption is the power consumption consumed during the third time period.

[0124] Based on the power consumption model of maintaining the processing frequency corresponding to Formula 3, the electronic device substitutes the third duration and the third probability into the power consumption model to obtain the third power consumption. For example, Formula 3: E de =f 3 *t(the third duration to keep the processing frequency unchanged)*P r (The third probability of keeping the processing frequency unchanged). The electronic device calculates t (the third time period of keeping the processing frequency unchanged) and P r Substituting the third probability of keeping the processing frequency unchanged into Formula 3 yields the power consumption as the third power consumption.

[0125] In one possible implementation, the third duration includes sub-durations corresponding to multiple stages, and the third probability includes sub-probabilities corresponding to multiple stages; accordingly, the steps for the electronic device to determine the third power consumption based on the third duration and the third probability may be: for any stage, the electronic device determines the third sub-power consumption corresponding to the stage based on the sub-duration corresponding to the stage and the sub-probability corresponding to the stage; and the third sub-power consumptions corresponding to multiple stages are accumulated to obtain the third power consumption.

[0126] The electronic device determines E based on formula 3 and the data in Table 1. de0 =0;E de1 =0;E de2 =0.0102f 3 ;E de3 =0.3851f 3 , then the third power consumption can be E de0 +E de1 +E de2 +E de3 =0.7853f 3 .

[0127] (4) The electronic device determines a fourth power consumption based on the total duration obtained by accumulating the first duration, the second duration, and the third duration. The fourth power consumption is the power consumption consumed by the total duration of the historical rendering process maintaining the processing frequency unchanged.

[0128] Based on the power consumption model of maintaining the processing frequency corresponding to Formula 4, the electronic device substitutes the total duration into the power consumption model to obtain the fourth power consumption. For example, Formula 4: E cg =f 3 *t(total time)*100%; the electronic device substitutes t(total time) into formula 4 to obtain the power consumption, which is the fourth power consumption.

[0129] In one possible implementation, the total duration includes sub-durations corresponding to multiple stages; accordingly, the electronic device determines the fourth power consumption based on the total duration obtained by accumulating the first, second, and third durations. The steps may be: for any stage, the electronic device determines the fourth sub-power consumption corresponding to the stage based on the sub-duration corresponding to the stage; and accumulates the fourth sub-power consumptions corresponding to the multiple stages to obtain the fourth power consumption. For example, based on Formula 4 and the data in Table 1, the electronic device determines the fourth power consumption to be E cg =17.2074f 3 .

[0130] (5) The electronic device determines the first relationship data based on the first power consumption relationship data, the second power consumption relationship data, the third power consumption, and the fourth power consumption.

[0131] Since the first power consumption relationship data, the second power consumption relationship data, the third power consumption and the fourth power consumption all include f3 , the electronic device will take the first power consumption relationship data, the second power consumption relationship data, the third power consumption and the fourth power consumption as f 3 Eliminate, combined with the power consumption constraint relationship: the fourth power consumption is greater than or equal to the sum of the first power consumption constrained by the first power consumption relationship data, the second power consumption constrained by the second power consumption relationship data, and the third power consumption; that is, E cg >=E up +E down +E de The final first relationship data is 0.2642a 2 +16.1108b 2 <=16.4221; where a represents the second adjustment coefficient, and b represents the third adjustment coefficient; and the value range of a is (1, +∞), and the value range of b is (0, 1).

[0132] Correspondingly, the steps for the electronic device to determine the first relationship data based on the first power consumption relationship data, the second power consumption relationship data, the third power consumption and the fourth power consumption may be: the electronic device accumulates the first power consumption, the second power consumption and the third power consumption based on the first power consumption relationship data and the second power consumption relationship data to obtain the third power consumption relationship data; and determines the first relationship data based on the third power consumption relationship data and the fourth power consumption, and the first relationship data satisfies that the power consumption represented by the third power consumption relationship data is greater than the fourth power consumption.

[0133] For example, the first relation data is 0.2642a 2 +16.1108b 2 <=16.4221; wherein a represents the second adjustment coefficient, b represents the third adjustment coefficient, the value range of b is (0, 1), and the value range of a is (1, +∞); the electronic device converts the first relationship data into a two-dimensional image with a as the horizontal coordinate and b as the vertical coordinate, such as Figure 10 shown.

[0134] The process of determining the second relational data:

[0135] In order to ensure the user experience, electronic devices need to strictly control frame loss; therefore, it is expected that the rendering time of each frame image will be controlled within 80% of the frame period; since the rendering process includes two main threads, the UI thread and the Render thread; therefore, t UI +t Render ≤0.8T; where t UI Including the duration of the first and second stages respectively; t RenderIncluding the duration corresponding to the third stage, T represents the frame period; that is, the total duration of the first three stages must be less than 80% of the frame period; the mathematical expectation of the frame period is: T = t0 + t1 + t2 + t3; where t0 represents the duration of the first stage, t1 represents the duration of the second stage, t2 represents the duration of the third stage, and t3 represents the duration of the fourth stage; based on the data in Table 1, it can be determined that T = 1.6433 + 2.285 + 3.9244 + 9.3547 = 17.2074. The time expectation after the electronic device adjusts the processing frequency based on the second adjustment coefficient and the third adjustment coefficient is: Et = t (the first duration of increasing the processing frequency) / a*P r (the first probability of increasing the processing frequency) + t (the second duration of decreasing the processing frequency) / b*P r (the second probability of reducing the processing frequency) + t (the third duration of keeping the processing frequency unchanged) * P r (The third probability of keeping the processing frequency unchanged); Based on the data in Table 1, the electronic device can derive the second relationship data as 0.2642 / a+7.1411 / b<=13.3658; wherein, a represents the second adjustment coefficient, b represents the third adjustment coefficient, the value range of b is (0, 1), and the value range of a is (1, +∞).

[0136] The historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process; accordingly, the electronic device determines the second relationship data based on the historical rendering data through the following steps (A) to (E), including:

[0137] (A) The electronic device determines first performance relationship data based on the first duration and the first probability, where the first performance relationship data is used to represent the relationship between the second adjustment coefficient and the first performance corresponding to the first duration.

[0138] The electronic device stores in advance a first performance model corresponding to increasing the processing frequency. The first performance model is the relationship between duration, probability and performance, and the duration in the first performance model = first duration / second adjustment coefficient; therefore, the electronic device substitutes the first duration and the first probability into the first performance model, and the obtained relationship data is the first performance relationship data.

[0139] (B) The electronic device determines second performance relationship data based on the second duration and the second probability, where the second performance relationship data is used to represent the relationship between the third adjustment coefficient and the second performance corresponding to the second duration.

[0140] The electronic device stores in advance a second performance model corresponding to the reduction in processing frequency. The second performance model is the relationship between duration, probability and performance, and the duration in the second performance model = second duration / third adjustment coefficient; therefore, the electronic device substitutes the second duration and the second probability into the second performance model, and the obtained relationship data is the second performance relationship data.

[0141] (C) The electronic device determines a third performance based on the third duration and the third probability, where the third performance is the performance corresponding to the third duration.

[0142] The electronic device stores in advance a third performance model that keeps the processing frequency unchanged. The third performance model is the relationship between duration, probability and performance, and the duration in the third performance model is the third duration; therefore, the performance obtained by the electronic device by substituting the third duration and the third probability into the third performance model is the third performance.

[0143] (D) The electronic device determines a fourth performance based on the total duration obtained by accumulating the first duration, the second duration, and the third duration. The fourth performance is the performance corresponding to the total duration in which the processing frequency remains unchanged in the historical rendering process.

[0144] The electronic device stores in advance a fourth performance model that keeps the processing frequency unchanged. The fourth performance model is the relationship between duration and performance, and the duration in the fourth performance model is the total duration of the first duration, the second duration and the third duration; therefore, the performance obtained by the electronic device when the total duration is substituted into the fourth performance model is the fourth performance.

[0145] (E) The electronic device determines second relationship data based on the first performance relationship data, the second performance relationship data, the third performance, and the fourth performance.

[0146] Based on the first performance relationship data and the second performance relationship data, the electronic device accumulates the first performance, the second performance and the third performance to obtain the third performance relationship data; based on the third performance relationship data and the fourth performance, the electronic device determines the second relationship data, and the performance represented by the second relationship data satisfying the third performance relationship is greater than the fourth performance.

[0147] For example, the second relationship data is 0.2642 / a+7.1411 / b<=13.3658; wherein a represents the second adjustment coefficient, b represents the third adjustment coefficient, the value range of b is (0, 1), and the value range of a is (1, +∞); the electronic device converts the second relationship data into a two-dimensional image with a as the horizontal coordinate and b as the vertical coordinate, such as Figure 11 The electronic device will Figure 10 The two-dimensional image and Figure 11 The target relationship data obtained by intersecting the two-dimensional images shown is as follows Figure 12 shown.

[0148] In an embodiment of the present application, the electronic device establishes a frame rendering constraint model based on a power model and a performance model; when adjusting the processing frequency, it can not only meet the basic performance requirements, but also release redundant power consumption space, thereby achieving relatively ideal power consumption benefits, increasing the usage time of the electronic device, and thus improving the user experience.

[0149] The third situation: the current stage is the third stage in the rendering process, the target stage is the third stage, and the third stage is the Render thread execution stage; when the actual completion time of the UI thread is earlier than the expected time, the processing frequency can be reduced; because the risk of frame loss is much lower than expected when the actual completion time of the UI thread is earlier; therefore, the processing frequency can be reduced, thereby saving power consumption of the electronic device; when the actual completion time of the UI thread is later than the expected time, the processing frequency can be increased, thereby reducing the occurrence of frame loss. In addition, the electronic device can determine the adjustment coefficient based on the proportion of the thread completion time of the UI thread to the entire frame. Accordingly, the steps for the electronic device to determine the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy can be:

[0150] When the frequency adjustment strategy is used to represent an increase in the processing frequency or a decrease in the processing frequency, the electronic device determines the thread completion time of the UI thread and the expected rendering time of the current frame image; determines the ratio of the thread completion time of the UI thread to the expected rendering time of the current frame image, and obtains the fifth adjustment coefficient of the third stage.

[0151] The fourth situation: the current stage is the fourth stage in the rendering process, the target stage is the first stage of the next frame image, the fourth stage is the remaining time stage after the Render thread completes the rendering stage and before receiving the VSYNC signal of the next frame image, and the first stage of the next frame image is the waiting stage before the UI thread executes after receiving the VSYNC signal of the next frame image; accordingly, the electronic device determines the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy, which can be: the electronic device determines the fourth adjustment coefficient of the first stage of the next frame image based on the actual rendering completion time of the current frame image and the frequency adjustment strategy.

[0152] The steps for the electronic device to determine the fourth adjustment coefficient of the first stage of the next frame image based on the actual rendering completion time and frequency adjustment strategy of the current frame image may be: the electronic device determines the actual rendering duration of the current frame image based on the actual rendering completion time of the current frame image and the reception time of the VSYNC signal; determines the actual duration coefficient of the current frame image based on the actual rendering duration and the expected rendering duration of the current frame image; determines the fourth adjustment coefficient of the first stage of the next frame image based on the actual duration coefficient of the current frame image and the frequency adjustment strategy.

[0153] For example, the electronic device may determine the actual rendering duration of the current frame image based on the actual rendering completion time of the current frame image and the reception time of the VSYNC signal as follows: the electronic device determines the time difference between the actual rendering completion time of the current frame image and the reception time of the VSYNC signal to obtain the actual rendering duration of the current frame image.

[0154] For example, the electronic device may determine the actual duration coefficient of the current frame image based on the actual rendering duration and the expected rendering duration of the current frame image by: the electronic device determines the ratio between the actual rendering duration and the expected rendering duration of the current frame image to obtain the actual duration coefficient of the current frame image.

[0155] In order to control the CPU processing frequency from being reduced too low or increased too high, the electronic device sets an upper limit coefficient and a lower limit coefficient to constrain the fourth adjustment coefficient, thereby avoiding the risk of frame loss caused by the CPU processing frequency being too low, or excessive power consumption caused by the CPU processing frequency being too high. Accordingly, the electronic device can determine the fourth adjustment coefficient of the first stage of the next frame image based on the actual duration coefficient of the current frame image and the frequency adjustment strategy by:

[0156] When the frequency adjustment strategy is used to represent an increase in the processing frequency, the electronic device determines the fourth adjustment coefficient of the first stage of the next frame image based on the actual duration coefficient of the current frame image and the first preset duration coefficient, and the fourth adjustment coefficient is the minimum duration coefficient of the actual duration coefficient of the current frame image and the first preset duration coefficient; when the frequency adjustment strategy is used to represent a reduction in the processing frequency, the electronic device determines the fourth adjustment coefficient of the first stage of the next frame image based on the actual duration coefficient of the current frame image and the second preset duration coefficient, and the fourth adjustment coefficient is the maximum duration coefficient of the actual duration coefficient of the previous frame image and the second preset duration coefficient, and the first preset duration coefficient is greater than the second preset duration coefficient.

[0157] The first preset duration coefficient is the upper limit coefficient; the second preset duration coefficient is the lower limit coefficient; and, both the first preset duration coefficient and the second preset duration coefficient can be set and changed as needed; in the embodiment of the present application, there is no specific limitation on the first preset duration coefficient and the second preset duration coefficient; for example, the first preset duration coefficient can be 1.2 or 1.25; the second preset duration coefficient can be 0.75 or 0.8, etc.

[0158] In one possible implementation, the electronic device may provide a reference for the frame loss prediction information of the next frame image of the current frame image based on the actual rendering completion time of the current frame image; accordingly, the process may be: when the actual rendering duration of the current frame image is less than the fifth preset duration, the electronic device determines the frequency adjustment flag of the current frame image as a reduction flag; when the actual rendering duration of the current frame image is greater than the fifth preset duration, the electronic device determines the frequency adjustment flag of the current frame image as an increase flag, and the frequency adjustment flag of the current frame image is used to determine the frame loss prediction information of the next frame image of the current frame image.

[0159] The fifth preset duration may be determined by: the electronic device determining the expected rendering duration of the current frame image; determining the fifth preset duration based on the expected rendering duration of the current frame image and the third preset coefficient, wherein the fifth preset duration is less than the expected rendering duration of the current frame image; for example, determining the product of the expected rendering duration of the current frame image and the third preset coefficient to obtain the fifth preset duration. The third preset coefficient can be set and changed as needed. In the embodiment of the present application, the third preset coefficient is not specifically limited; for example, the third preset coefficient may be 75%, 80%, or 85%.

[0160] In another possible implementation, the electronic device may directly determine the fourth adjustment coefficient based on the proportion of the actual rendering completion time of the current frame image to the entire frame; accordingly, the electronic device may determine the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy as follows: when the frequency adjustment strategy is used to represent an increase in the processing frequency or to represent a decrease in the processing frequency, the electronic device determines the actual rendering time and the expected rendering time of the current frame image; based on the ratio of the actual rendering time of the current frame image to the expected rendering time, obtain the fourth adjustment coefficient of the fourth stage. For example, continue to refer to Figure 9 , the electronic device can adjust the CPU load based on the proportion of the time taken to complete the current frame image to the entire frame.

[0161] Step 304: The electronic device adjusts the processing frequency of the target stage based on the adjustment coefficient.

[0162] In one possible implementation, the electronic device scales the rendering load value based on the adjustment coefficient to adjust the processing frequency. Accordingly, the electronic device may adjust the processing frequency of the target stage based on the adjustment coefficient by:

[0163] The electronic device adjusts the first rendering load value based on the adjustment coefficient to obtain a second rendering load value, and adjusts the processing frequency of the target stage based on the second rendering load value; for example, the corresponding processing frequency is allocated to the target stage based on the second rendering load value.

[0164] In another possible implementation, the electronic device scales the processing frequency before adjustment based on the adjustment coefficient to adjust the processing frequency; accordingly, the step of the electronic device adjusting the processing frequency of the target stage based on the adjustment coefficient may be: the electronic device adjusts the processing frequency of the target stage based on the adjustment coefficient; for example, if the adjustment coefficient is 1.5, the adjusted processing frequency is 1.5 times the processing frequency before adjustment.

[0165] It should be noted that after determining the first adjustment coefficient of the first stage, the electronic device adjusts the processing frequency of the first stage based on the first adjustment coefficient of the first stage. The adjusted processing frequency continues until the start of the second stage. Thereafter, the processing frequency of the second stage is adjusted based on the second adjustment coefficient or the third adjustment coefficient of the second stage. The adjusted processing frequency continues until the start of the third stage, and then the processing frequency of the third stage is adjusted based on the fifth adjustment coefficient of the third stage.

[0166] In one possible implementation, when the frequency adjustment strategy of the third stage is used to represent an increase in the processing frequency, the electronic device may continue to use the second adjustment coefficient of the second stage to adjust the processing frequency of the third stage; or update the second adjustment coefficient to a fifth adjustment coefficient and adjust the processing frequency of the third stage based on the fifth adjustment coefficient. When the frequency adjustment strategy of the third stage is used to represent a decrease in the processing frequency, the electronic device may continue to use the third adjustment coefficient of the second stage to adjust the processing frequency of the third stage; or update the third adjustment coefficient to the fifth adjustment coefficient and adjust the processing frequency of the third stage based on the fifth adjustment coefficient.

[0167] To verify the effectiveness of the embodiments of the present application on electronic devices, the processing frequency adjustment mechanism provided by the embodiments of the present application was deployed on an electronic device running a Snapdragon 8gen2 chip for testing. The test results show that for quickly rendered frame images, by reducing their corresponding processing frequency, they can still be completed before the expected deadline, thereby meeting the performance requirements of the electronic device while reducing the power consumption of the electronic device, thereby reducing unnecessary overhead. For example, while the performance of the electronic device does not significantly decline (the average frame loss rate fluctuates by less than 1%), the power consumption of the electronic device can be reduced by 3% to 7%.

[0168] refer to Figure 13 , which shows a structural block diagram of a frequency adjustment device for a rendering process provided by an exemplary embodiment of the present application. The device includes:

[0169] A first determining module 1301 is configured to determine, for a current stage in a rendering process of a current frame image, frame loss prediction information of the current stage, wherein the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image;

[0170] A second determining module 1302 is configured to determine a frequency adjustment strategy based on the frame loss prediction information, where the frequency adjustment strategy is used to represent an adjustment method for a processing frequency;

[0171] The adjustment module 1303 is configured to adjust the processing frequency of a target stage based on the frequency adjustment strategy, where the target stage is the stage in the rendering process of the current stage or the next frame image of the current frame image.

[0172] In one possible implementation, the second determination module 1302 is configured to determine that the frequency adjustment strategy is used to represent an increase in processing frequency when the frame loss prediction information is used to predict that frame loss will occur in the current frame image during the rendering process; and to determine that the frequency adjustment strategy is used to represent a decrease in processing frequency when the frame loss prediction information is used to predict that frame loss will not occur in the current frame image during the rendering process.

[0173] In another possible implementation, the current stage is the first stage in the rendering process, and the first stage is the waiting stage before the user interface UI thread is executed after receiving the vertical synchronization signal; the first determination module 1301 is used to determine the frequency adjustment identifier of the previous frame image of the current frame image; when the frequency adjustment identifier is a reduction identifier, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the frequency adjustment identifier is an increase identifier, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process.

[0174] In another possible implementation, the first determination module 1301 is used to determine the actual rendering time of the previous frame image; when the actual rendering time of the previous frame image is less than the first preset time, determine that the frequency adjustment flag of the previous frame image is a reduction flag; when the actual rendering time of the previous frame image is greater than the first preset time, determine that the frequency adjustment flag of the previous frame image is an increase flag.

[0175] In another possible implementation, the current stage is the second stage in the rendering process, and the second stage is the UI thread execution stage; the first determination module 1301 is used to determine the execution delay of the UI thread, and the execution delay is the time difference between the execution start time of the UI thread and the reception time of the vertical synchronization signal; when the execution delay is less than the second preset duration, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the execution delay is greater than the third preset duration, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process, and the third preset duration is greater than the second preset duration.

[0176] In another possible implementation, the current stage is the third stage in the rendering process, and the third stage is the rendering thread execution stage; the first determination module 1301 is used to determine the thread completion time of the previous stage of the third stage; when the thread completion time is less than a fourth preset time, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the thread completion time is greater than the fourth preset time, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process.

[0177] In another possible implementation, the apparatus further includes:

[0178] A third determining module is used to determine the expected rendering time of the current frame image;

[0179] The fourth determining module is configured to determine the fourth preset duration based on the expected rendering duration and a preset coefficient, where the fourth preset duration is less than the expected rendering duration.

[0180] In another possible implementation, the current stage is the fourth stage in the rendering process, and the fourth stage is the remaining time stage after the Render thread completes rendering and before receiving the vertical synchronization signal of the next frame image; the first determination module 1301 is used to determine the actual rendering time of the current frame image; when the actual rendering time is less than the fifth preset time, the frame loss prediction information is determined to be the first prediction information, and the first prediction information is used to predict that the current frame image will not suffer frame loss in the rendering process; when the actual rendering time is greater than the fifth preset time, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process.

[0181] In another possible implementation, the adjustment module 1303 is configured to determine an adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy; and adjust the processing frequency of the target stage based on the adjustment coefficient.

[0182] In another possible implementation, the current stage is the first stage in the rendering process, the target stage is the first stage, and the first stage is a waiting stage before the UI thread is executed after receiving a vertical synchronization signal;

[0183] The adjustment module 1303 is configured to determine a first adjustment coefficient of the first stage based on an actual rendering completion time of a previous frame image of the current frame image and the frequency adjustment strategy.

[0184] In another possible implementation, the adjustment module 1303 is used to determine the actual rendering duration of the previous frame image based on the actual rendering completion time of the previous frame image and the reception time of the vertical synchronization signal of the previous frame image; determine the actual duration coefficient of the previous frame image based on the actual rendering duration and the expected rendering duration of the previous frame image; and determine the first adjustment coefficient of the first stage based on the actual duration coefficient of the previous frame image and the frequency adjustment strategy.

[0185] In another possible implementation, the adjustment module 1303 is used to determine the first adjustment coefficient of the first stage based on the actual duration coefficient and the first preset duration coefficient when the frequency adjustment strategy is used to represent an increase in the processing frequency, and the first adjustment coefficient is the minimum duration coefficient of the actual duration coefficient and the first preset duration coefficient; when the frequency adjustment strategy is used to represent a reduction in the processing frequency, the first adjustment coefficient of the first stage is determined based on the actual duration coefficient and the second preset duration coefficient, and the first adjustment coefficient is the maximum duration coefficient of the actual duration coefficient and the second preset duration coefficient, and the first preset duration coefficient is greater than the second preset duration coefficient.

[0186] In another possible implementation, the current stage is the second stage in the rendering process, the target stage is the second stage, and the second stage is the UI thread execution stage;

[0187] The adjustment module 1303 is used to determine the second adjustment coefficient of the second stage when the frequency adjustment strategy is used to represent an increase in the processing frequency; when the frequency adjustment strategy is used to represent a reduction in the processing frequency, determine the third adjustment coefficient of the second stage, the second adjustment coefficient and the third adjustment coefficient satisfy the target relationship data, and the target relationship data is used to represent the relationship between power consumption and performance.

[0188] In another possible implementation, the apparatus further includes:

[0189] a fifth determining module, configured to determine first relationship data based on historical rendering data, wherein the first relationship data is used to represent a power consumption relationship satisfied by the second adjustment coefficient and the third adjustment coefficient;

[0190] a sixth determining module, configured to determine second relationship data based on the historical rendering data, wherein the second relationship data is used to characterize a performance relationship satisfied by the second adjustment coefficient and the third adjustment coefficient;

[0191] A seventh determining module is configured to determine the target relationship data based on the first relationship data and the second relationship data.

[0192] In another possible implementation, the historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process;

[0193] The fifth determination module is used to determine first power consumption relationship data based on the first duration and the first probability, where the first power consumption relationship data is used to characterize the relationship between the second adjustment coefficient and the first power consumption consumed by the first duration; determine second power consumption relationship data based on the second duration and the second probability, where the second power consumption relationship data is used to characterize the relationship between the third adjustment coefficient and the second power consumption consumed by the second duration; determine third power consumption based on the third duration and the third probability, where the third power consumption is the power consumption consumed by the third duration; determine fourth power consumption based on the total duration obtained by adding the first duration, the second duration, and the third duration, where the fourth power consumption is the power consumption consumed by the total duration when the historical rendering process maintains the processing frequency unchanged; determine the first relationship data based on the first power consumption relationship data, the second power consumption relationship data, the third power consumption, and the fourth power consumption.

[0194] In another possible implementation, the first duration includes sub-durations corresponding to a plurality of stages, and the first probability includes sub-probabilities corresponding to the plurality of stages.

[0195] The fifth determination module is used to determine, for any stage, the first sub-power consumption relationship data corresponding to the stage based on the sub-duration corresponding to the stage and the sub-probability corresponding to the stage; and accumulate the first sub-power consumption relationship data corresponding to the multiple stages to obtain the first power consumption relationship data.

[0196] In another possible implementation, the fifth determination module is used to accumulate the first power consumption, the second power consumption and the third power consumption based on the first power consumption relationship data and the second power consumption relationship data to obtain third power consumption relationship data; and determine the first relationship data based on the third power consumption relationship data and the fourth power consumption, and the first relationship data satisfies that the power consumption represented by the third power consumption relationship data is greater than the fourth power consumption.

[0197] In another possible implementation, the historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process;

[0198] The sixth determination module is used to determine first performance relationship data based on the first duration and the first probability, where the first performance relationship data is used to characterize the relationship between the second adjustment coefficient and the first performance corresponding to the first duration; determine second performance relationship data based on the second duration and the second probability, where the second performance relationship data is used to characterize the relationship between the third adjustment coefficient and the second performance corresponding to the second duration; determine third performance based on the third duration and the third probability, where the third performance is the performance corresponding to the third duration; determine fourth performance based on the total duration obtained by adding the first duration, the second duration, and the third duration, where the fourth performance is the performance corresponding to the total duration in the historical rendering process that keeps the processing frequency unchanged; determine the second relationship data based on the first performance relationship data, the second performance relationship data, the third performance, and the fourth performance.

[0199] In another possible implementation, the sixth determination module is used to accumulate the first performance, the second performance and the third performance based on the first performance relationship data and the second performance relationship data to obtain third performance relationship data; and determine the second relationship data based on the third performance relationship data and the fourth performance, and the second relationship data satisfies the performance represented by the third performance relationship greater than the fourth performance.

[0200] In another possible implementation, the current stage is the fourth stage in the rendering process, the target stage is the first stage of the next frame image, the fourth stage is the remaining time stage after the Render thread completes the rendering stage and before receiving the vertical synchronization signal of the next frame image, and the first stage of the next frame image is the waiting stage before the UI thread executes after receiving the vertical synchronization signal of the next frame image;

[0201] The adjustment module 1303 is configured to determine a fourth adjustment coefficient of the first stage of the next frame image based on the actual rendering completion time of the current frame image and the frequency adjustment strategy.

[0202] In another possible implementation, the adjustment module 1303 is used to determine the actual rendering duration of the current frame image based on the actual rendering completion time of the current frame image and the reception time of the vertical synchronization signal; determine the actual duration coefficient of the current frame image based on the actual rendering duration and the expected rendering duration of the current frame image; and determine the fourth adjustment coefficient of the first stage of the next frame image based on the actual duration coefficient of the current frame image and the frequency adjustment strategy.

[0203] In another possible implementation, the apparatus further includes:

[0204] A seventh determining module is configured to determine, when the actual rendering duration of the current frame image is less than a fifth preset duration, that the frequency adjustment flag of the current frame image is a reduction flag;

[0205] The eighth determination module is used to determine that the frequency adjustment flag of the current frame image is an increase flag when the actual rendering time of the current frame image is greater than the fifth preset time. The frequency adjustment flag of the current frame image is used to determine the frame loss prediction information of the next frame image of the current frame image.

[0206] In another possible implementation, the adjustment module 1303 is configured to adjust the first rendering load value based on the adjustment coefficient to obtain a second rendering load value, and adjust the processing frequency of the target stage based on the second rendering load value.

[0207] In another possible implementation, the processing frequency is a processing frequency; the adjustment module 1303 is used to increase the processing frequency of the target stage when the frequency adjustment strategy is used to represent an increase in the processing frequency; and to reduce the processing frequency of the target stage when the frequency adjustment strategy is used to represent a decrease in the processing frequency.

[0208] In another possible implementation, the adjustment module 1303 is configured to, when the target stage is the second stage in the rendering process, increase the processing frequency of the CPU in the second stage, where the second stage is the UI thread execution stage;

[0209] The adjustment module 1303 is configured to increase the processing frequency of the CPU in the third stage when the target stage is the third stage in the rendering process. The third stage is the Render thread execution stage.

[0210] In an embodiment of the present application, the rendering process of the current frame image is divided into multiple stages. For the current stage, the processing frequency is adjusted based on the frame loss prediction information of the current stage. Therefore, through the stage-by-stage processing frequency adjustment, the processing frequency can be adjusted in a timely manner, which can improve the accuracy of the processing frequency adjustment.

[0211] It should be noted that the frequency adjustment device for the rendering process provided in the above embodiment only uses the division of the above functional modules as an example when performing frequency adjustment for the rendering process. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device can be divided into different functional modules to complete all or part of the functions described above. In addition, the frequency adjustment device for the rendering process provided in the above embodiment and the frequency adjustment method embodiment for the rendering process belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0212] See also Figure 14 , Figure 14 FIG1 is a schematic diagram of the structure of an electronic device 1400 provided by an exemplary embodiment of the present application. The electronic device 1400 may further include one or more of the following components: a processor 1410 , a memory 1420 , and a display screen 1430 .

[0213] The processor 1410 uses various interfaces and lines to connect the various parts of the entire electronic device 1400. By running or executing instructions, programs, code sets or instruction sets stored in the memory 1420, and calling data stored in the memory 1420, it performs various functions of the electronic device 1400 and processes data. Optionally, the processor 1410 can be implemented in the form of at least one hardware of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1410 can integrate one or a combination of a central processing unit (CPU) 14101, a graphics processing unit (GPU) 14102, a neural-network processing unit (NPU) 14103, and a modem 14104. The CPU 14101 primarily handles the operating system, user interface, and application programs; the GPU 14102 is responsible for rendering and drawing content displayed on the display screen 1430; the NPU 14103 is used to implement artificial intelligence (AI) functions; and the modem 14104 is used to handle wireless communications. It is understood that the modem can also be implemented independently of the processor 1410 via a separate computer program product.

[0214] The memory 1420 may include a random access memory (RAM) 14201, and may also include a read-only memory (ROM) 14202. Optionally, the memory 1420 includes a non-transitory computer-readable storage medium 14203. The memory 1420 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device 1400 (such as audio data, a phone book), etc.

[0215] The display screen 1430 is a display component for displaying a user interface. Optionally, the display screen 1430 is a display screen with a touch function, through which the user can use any suitable object such as a finger or a touch pen to perform touch operations on the display screen 1430.

[0216] Display screen 1430 is typically provided on the front panel of electronic device 1400. Display screen 1430 can be designed as a full-screen, curved screen, special-shaped screen, double-sided screen, or foldable screen. Display screen 1430 can also be designed as a combination of a full-screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which are not limited in this embodiment.

[0217] In addition, those skilled in the art will appreciate that the structure of electronic device 1400 shown in the above figures does not limit electronic device 1400. Electronic device 1400 may include more or fewer components than shown, or may combine certain components, or arrange components differently. For example, electronic device 1400 may also include a WiFi module, an audio acquisition device, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a Bluetooth module, a power supply, and other components, which will not be described in detail here.

[0218] An embodiment of the present application provides a chip, which includes a programmable logic circuit and / or program instructions. When the chip runs on an electronic device, the chip is used to implement the frequency adjustment method for the rendering process as described in the above embodiment.

[0219] An embodiment of the present application provides a computer-readable storage medium storing at least one computer instruction, wherein the at least one computer instruction is configured to be executed by a processor to implement the frequency adjustment method for a rendering process as described in the above embodiment.

[0220] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the frequency adjustment method for the rendering process as described in the above embodiment.

[0221] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0222] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A frequency adjustment method for a rendering process, characterized in that: The method comprises: For a current stage in a rendering process of a current frame image, determining frame loss prediction information of the current stage, wherein the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image; Determining a frequency adjustment strategy based on the frame loss prediction information, wherein the frequency adjustment strategy is used to characterize an adjustment method of a processing frequency; Based on the frequency adjustment strategy, the processing frequency of the target stage is adjusted, and the target stage is a stage in the rendering process of the current stage or the next frame image of the current frame image.

2. The method according to claim 1, characterized in that The determining of a frequency adjustment strategy based on the frame loss prediction information includes: In a case where the frame loss prediction information is used to predict that the current frame image will suffer frame loss during the rendering process, determining the frequency adjustment strategy to represent an increase in processing frequency; In a case where the frame loss prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process, the frequency adjustment strategy is determined to represent a reduction in processing frequency.

3. The method according to claim 1, characterized in that The current stage is the first stage in the rendering process, which is a waiting stage before the user interface UI thread is executed after receiving a vertical synchronization signal; determining the frame loss prediction information of the current stage includes: Determining a frequency adjustment flag of a previous frame image of the current frame image; When the frequency adjustment flag is a reduction flag, determining that the frame loss prediction information is first prediction information, where the first prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process; When the frequency adjustment flag is an increase flag, the frame loss prediction information is determined to be second prediction information, and the second prediction information is used to predict that frame loss will occur in the current frame image during the rendering process.

4. The method according to claim 3, characterized in that The determining of the frequency adjustment identifier of the previous frame image of the current frame image includes: Determine the actual rendering time of the previous frame of image; When the actual rendering duration of the previous frame of image is less than the first preset duration, determining the frequency adjustment flag of the previous frame of image to be a reduction flag; When the actual rendering time of the previous frame image is greater than the first preset time, the frequency adjustment flag of the previous frame image is determined to be an increase flag.

5. The method according to claim 1, wherein The current stage is the second stage in the rendering process, and the second stage is the UI thread execution stage; and determining the frame loss prediction information of the current stage includes: Determine an execution delay of the UI thread, where the execution delay is a time difference between an execution start time of the UI thread and a reception time of a vertical synchronization signal; When the execution delay is less than a second preset time length, determining that the frame loss prediction information is first prediction information, where the first prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process; When the execution delay is greater than the third preset duration, the frame loss prediction information is determined to be the second prediction information, and the second prediction information is used to predict that the current frame image will suffer frame loss in the rendering process, and the third preset duration is greater than the second preset duration.

6. The method according to claim 1, characterized in that The current stage is the third stage in the rendering process, and the third stage is the rendering thread execution stage; and determining the frame loss prediction information of the current stage includes: Determine the thread completion time of the previous stage of the third stage; When the thread completion time is less than a fourth preset time, determining that the frame loss prediction information is first prediction information, where the first prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process; When the thread completion time is longer than the fourth preset time, the frame loss prediction information is determined to be second prediction information, where the second prediction information is used to predict that frame loss will occur in the current frame image during the rendering process.

7. The method according to claim 6, characterized in that The method further comprises: Determining an expected rendering time of the current frame image; The fourth preset duration is determined based on the expected rendering duration and a preset coefficient, and the fourth preset duration is less than the expected rendering duration.

8. The method according to claim 1, characterized in that The current stage is the fourth stage in the rendering process, and the fourth stage is the remaining time stage after the Render thread completes rendering and before receiving the vertical synchronization signal of the next frame image; determining the frame loss prediction information of the current stage includes: Determine the actual rendering time of the current frame image; When the actual rendering duration is less than a fifth preset duration, determining that the frame loss prediction information is first prediction information, where the first prediction information is used to predict that no frame loss will occur in the current frame image during the rendering process; When the actual rendering duration is greater than the fifth preset duration, the frame loss prediction information is determined to be second prediction information, and the second prediction information is used to predict that frame loss will occur in the current frame image during the rendering process.

9. The method according to any one of claims 1 to 8, characterized in that The adjusting the processing frequency of the target stage based on the frequency adjustment strategy includes: determining an adjustment coefficient for the target phase based on the current phase and the frequency adjustment strategy; The processing frequency of the target stage is adjusted based on the adjustment coefficient.

10. The method according to claim 9, characterized in that The current stage is the first stage in the rendering process, the target stage is the first stage, and the first stage is the waiting stage before the UI thread is executed after receiving the vertical synchronization signal; The determining the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy includes: A first adjustment coefficient of the first stage is determined based on an actual rendering completion time of a previous frame image of the current frame image and the frequency adjustment strategy.

11. The method according to claim 10, characterized in that The determining of the first adjustment coefficient of the first stage based on the actual rendering completion time of the previous frame image of the current frame image and the frequency adjustment strategy includes: Determining an actual rendering duration of the previous frame of image based on an actual rendering completion time of the previous frame of image and a reception time of a vertical synchronization signal of the previous frame of image; Determining an actual rendering duration coefficient of the previous frame image based on the actual rendering duration and the expected rendering duration of the previous frame image; Based on the actual duration coefficient of the previous frame image and the frequency adjustment strategy, a first adjustment coefficient of the first stage is determined.

12. The method according to claim 11, characterized in that The determining of the first adjustment coefficient of the first stage based on the actual duration coefficient of the previous frame image and the frequency adjustment strategy includes: In a case where the frequency adjustment strategy is used to represent an increase in the processing frequency, determining a first adjustment coefficient for the first stage based on the actual duration coefficient and the first preset duration coefficient, the first adjustment coefficient being the minimum duration coefficient of the actual duration coefficient and the first preset duration coefficient; In the case where the frequency adjustment strategy is used to characterize a reduction in processing frequency, a first adjustment coefficient of the first stage is determined based on the actual duration coefficient and the second preset duration coefficient, the first adjustment coefficient being the maximum duration coefficient between the actual duration coefficient and the second preset duration coefficient, and the first preset duration coefficient being greater than the second preset duration coefficient.

13. The method according to claim 9, characterized in that The current stage is the second stage in the rendering process, the target stage is the second stage, and the second stage is the UI thread execution stage; The determining the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy includes: determining a second adjustment coefficient for the second stage in a case where the frequency adjustment strategy is used to characterize an increase in processing frequency; When the frequency adjustment strategy is used to characterize a reduction in processing frequency, a third adjustment coefficient of the second stage is determined, the second adjustment coefficient and the third adjustment coefficient satisfy target relationship data, and the target relationship data is used to characterize the relationship between power consumption and performance.

14. The method according to claim 13, characterized in that The method further comprises: Determining first relationship data based on historical rendering data, where the first relationship data is used to characterize a power consumption relationship satisfied by the second adjustment coefficient and the third adjustment coefficient; determining, based on the historical rendering data, second relationship data for characterizing a performance relationship satisfied by the second adjustment coefficient and the third adjustment coefficient; The target relationship data is determined based on the first relationship data and the second relationship data.

15. The method according to claim 14, characterized in that The historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process; The determining of the first relationship data based on the historical rendering data includes: determining first power consumption relationship data based on the first duration and the first probability, where the first power consumption relationship data is used to represent a relationship between the second adjustment coefficient and first power consumption consumed during the first duration; determining second power consumption relationship data based on the second duration and the second probability, where the second power consumption relationship data is used to represent a relationship between the third adjustment coefficient and the second power consumption consumed during the second duration; determining a third power consumption based on the third duration and the third probability, where the third power consumption is the power consumption consumed during the third duration; Determining a fourth power consumption based on a total duration obtained by accumulating the first duration, the second duration, and the third duration, where the fourth power consumption is the power consumption consumed by the historical rendering process for the total duration when the processing frequency remains unchanged; The first relationship data is determined based on the first power consumption relationship data, the second power consumption relationship data, the third power consumption, and the fourth power consumption.

16. The method according to claim 15, characterized in that The first duration includes sub-durations corresponding to the multiple stages, and the first probability includes sub-probabilities corresponding to the multiple stages; The determining first power consumption relationship data based on the first duration and the first probability includes: For any stage, determining first sub-power consumption relationship data corresponding to the stage based on the sub-duration corresponding to the stage and the sub-probability corresponding to the stage; The first sub-power consumption relationship data corresponding to the multiple stages are accumulated to obtain the first power consumption relationship data.

17. The method according to claim 15, characterized in that The determining the first relationship data based on the first power consumption relationship data, the second power consumption relationship data, the third power consumption, and the fourth power consumption includes: Based on the first power consumption relationship data and the second power consumption relationship data, accumulating the first power consumption, the second power consumption and the third power consumption to obtain third power consumption relationship data; The first relationship data is determined based on the third power consumption relationship data and the fourth power consumption, wherein the first relationship data satisfies that the power consumption indicated by the third power consumption relationship data is greater than the fourth power consumption.

18. The method according to claim 14, characterized in that The historical rendering data includes a first duration corresponding to increasing the processing frequency, a first probability corresponding to increasing the processing frequency, a second duration corresponding to decreasing the processing frequency, a second probability corresponding to decreasing the processing frequency, a third duration corresponding to maintaining the processing frequency, and a third probability corresponding to maintaining the processing frequency in the historical rendering process; The determining of the second relationship data based on the historical rendering data includes: determining, based on the first duration and the first probability, first performance relationship data, where the first performance relationship data is used to characterize a relationship between the second adjustment coefficient and a first performance corresponding to the first duration; determining, based on the second duration and the second probability, second performance relationship data, where the second performance relationship data is used to characterize a relationship between the third adjustment coefficient and a second performance corresponding to the second duration; Determine a third performance based on the third duration and the third probability, where the third performance is the performance corresponding to the third duration; Determining a fourth performance based on a total duration obtained by summing the first duration, the second duration, and the third duration, where the fourth performance is a performance corresponding to the total duration when the processing frequency remains unchanged in the historical rendering process; The second relationship data is determined based on the first performance relationship data, the second performance relationship data, the third performance and the fourth performance.

19. The method according to claim 18, characterized in that The determining the second relationship data based on the first performance relationship data, the second performance relationship data, the third performance, and the fourth performance includes: Based on the first performance relationship data and the second performance relationship data, the first performance, the second performance and the third performance are accumulated to obtain third performance relationship data; The second relationship data is determined based on the third performance relationship data and the fourth performance, wherein the second relationship data satisfies the third performance relationship indicating that the performance is greater than the fourth performance.

20. The method according to claim 9, characterized in that The current stage is the fourth stage in the rendering process, the target stage is the first stage of the next frame image, the fourth stage is the remaining time stage after the Render thread completes the rendering stage and before receiving the vertical synchronization signal of the next frame image, and the first stage of the next frame image is the waiting stage before the UI thread executes after receiving the vertical synchronization signal of the next frame image; The determining the adjustment coefficient of the target stage based on the current stage and the frequency adjustment strategy includes: A fourth adjustment coefficient of the first stage of the next frame image is determined based on the actual rendering completion time of the current frame image and the frequency adjustment strategy.

21. The method according to claim 20, characterized in that The determining of the fourth adjustment coefficient of the first stage of the next frame image based on the actual rendering completion time of the current frame image and the frequency adjustment strategy includes: Determining an actual rendering duration of the current frame image based on an actual rendering completion time of the current frame image and a reception time of the vertical synchronization signal; Determining an actual rendering duration coefficient of the current frame image based on an actual rendering duration and an expected rendering duration of the current frame image; Based on the actual duration coefficient of the current frame image and the frequency adjustment strategy, a fourth adjustment coefficient of the first stage of the next frame image is determined.

22. The method according to claim 20, characterized in that The method further comprises: When the actual rendering duration of the current frame image is less than a fifth preset duration, determining that the frequency adjustment flag of the current frame image is a reduction flag; When the actual rendering time of the current frame image is greater than the fifth preset time, the frequency adjustment flag of the current frame image is determined to be an increase flag, and the frequency adjustment flag of the current frame image is used to determine the frame loss prediction information of the next frame image of the current frame image.

23. The method according to claim 9, characterized in that The adjusting the processing frequency of the target stage based on the adjustment coefficient includes: Adjusting the first rendering load value based on the adjustment coefficient to obtain a second rendering load value; Based on the second rendering load value, the processing frequency of the target stage is adjusted.

24. The method according to any one of claims 1 to 8, characterized in that The adjusting the processing frequency of the target stage based on the frequency adjustment strategy includes: In a case where the frequency adjustment strategy is used to represent an increase in processing frequency, increasing the processing frequency of the target stage; In a case where the frequency adjustment strategy is used to represent a reduction in processing frequency, the processing frequency of the target stage is reduced.

25. The method according to claim 24, characterized in that Increasing the processing frequency of the target stage includes: When the target stage is the second stage in the rendering process, increasing the processing frequency of the central processing unit (CPU) in the second stage, where the second stage is the UI thread execution stage; When the target stage is the third stage in the rendering process, the processing frequency of the CPU in the third stage is increased, and the third stage is the Render thread execution stage.

26. A frequency adjustment device for a rendering process, characterized in that: The device comprises: A first determining module is configured to determine, for a current stage in a rendering process of a current frame image, frame loss prediction information of the current stage, wherein the frame loss prediction information is used to predict whether frame loss will occur in the rendering process of the current frame image; A second determining module is configured to determine a frequency adjustment strategy based on the frame loss prediction information, wherein the frequency adjustment strategy is used to characterize an adjustment method of a processing frequency; An adjustment module is configured to adjust the processing frequency of a target stage based on the frequency adjustment strategy, where the target stage is a stage in a rendering process of the current stage or a next frame image of the current frame image.

27. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the frequency adjustment method for rendering process as described in any one of claims 1 to 25.

28. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the frequency adjustment method for rendering process according to any one of claims 1 to 25.

29. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the frequency adjustment method for rendering process as described in any one of claims 1 to 25.

Images