Dynamic frequency extraction method of CPU, electronic equipment and storage medium

Abstract

The invention relates to the technical field of terminal equipment, in particular to a dynamic frequency increasing method of a CPU, electronic equipment and a storage medium. The method comprises the following steps: detecting whether a current frame is lost or not, wherein the current frame is an image frame to be displayed on a display screen at the current moment; and under the condition that the current frame is lost, when the duration of synthesizing a target frame by the CPU is greater than or equal to a first threshold value, the working frequency of the CPU is improved to a specified frequency, and the target frame is the currently synthesized image frame. According to the dynamic frequency increasing method of the CPU, the electronic equipment and the storage medium provided by the invention, when the current frame is lost, the working frequency of the CPU is increased to the specified frequency. Therefore, the duration of the subsequent image frame on the CPU in the synthesis process can be shortened by improving the working frequency of the CPU, so that the synthesis time of the subsequent image frame is shortened, the possibility of frame loss of the subsequent image frame is reduced, smooth refreshing of a display picture is realized, and the use experience of a user is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminal devices, and particularly to a method for dynamically adjusting the frequency of a CPU, an electronic device, and a storage medium. Background Art

[0002] The display screen of an electronic device is used to display images, and the images displayed on the display screen can be updated under specified conditions. When the display image of the display screen is updated, in fact, the image frames displayed on the display screen are continuously refreshed. However, during the process of refreshing the image frames, there may occasionally be a stuttering phenomenon.

[0003] When refreshing the image frames, it is often necessary to go through the processes of drawing, rendering, composition, and display within a specified time. Among them, the composition thread includes a composition thread running on the Central Processing Unit (CPU) and a composition thread running on the Graphics Processing Unit (GPU). When the composition thread runs on a low-power CPU, it may cause the composition thread to time out, resulting in a stuttering phenomenon in the images displayed on the display screen, affecting the user experience. Summary of the Invention

[0004] To solve the above problems, this application provides a method for dynamically adjusting the frequency of a CPU, an electronic device, and a storage medium, which can set a reasonable scheduling strategy for a low-power CPU and reduce the possibility of frame loss.

[0005] To achieve the above object, in a first aspect, this application provides a method for dynamically adjusting the frequency of a CPU, including: detecting whether a current frame is lost, where the current frame is the image frame to be displayed on the display screen at the current moment; in the case where the current frame is lost, when the duration for the CPU to synthesize the target frame is greater than or equal to a first threshold, increasing the working frequency of the CPU to a specified frequency, where the target frame is the image frame currently being synthesized.

[0006] The method for dynamically adjusting the frequency of the CPU provided by this application can timely grasp the frame loss situation during the normal image frame update process of the electronic device. In the case where the current frame is lost, when the duration for the CPU to synthesize the target frame is greater than or equal to a first threshold, the working frequency of the CPU is increased to a specified frequency. In this way, by increasing the working frequency of the CPU, the duration for the CPU to synthesize the target frame can be reduced, so as to shorten the synthesis time of the target frame, thereby reducing the possibility of frame loss of the target frame, achieving smooth refreshing of the display screen, and improving the user experience.

[0007] In an alternative implementation, when a frame is dropped in the current frame, if the duration for the CPU to synthesize the target frame is greater than or equal to the first threshold, increase the working frequency of the CPU to a specified frequency, including: when a frame is dropped in the current frame, calculate the first threshold based on the duration for the CPU to synthesize historical frames; monitor the duration for the Render Engine thread to call the CPU to synthesize the target frame; when the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, increase the working frequency of the CPU to the specified frequency. In this way, after a frame-drop phenomenon occurs, the target frame can be adjusted in a timely manner. By adjusting the working frequency of the CPU, the duration for the CPU to synthesize the target frame can be adjusted, thereby reducing the possibility of frame drops in the target frame. At the same time, the adjustment of the CPU working frequency can be performed for a specific frame, without the CPU having to run at a high working frequency throughout the entire refresh process, effectively reducing the power consumption of the CPU.

[0008] In an alternative implementation, when a frame is dropped in the current frame, calculating the first threshold based on the duration for the CPU to synthesize historical frames includes: when a frame is dropped in the current frame, detect the refresh rate of the current image frame; determine the target historical frame according to the refresh rate of the current image frame; calculate an initial threshold based on the duration for the Render Engine thread to call the CPU to synthesize the target historical frame; determine whether the initial threshold is within the set threshold range; if the initial threshold meets the set threshold range, use the initial threshold as the first threshold. Since different refresh rates have different historical frames that have a greater impact on the current frame, it is necessary to select a specified target historical frame from the historical frames according to the refresh rate to facilitate obtaining a suitable first threshold later. After determining the target historical frame, the initial threshold can be calculated based on the duration for the Render Engine thread to call the CPU to synthesize the target historical frame, so as to use the initial threshold as the adjustment reference for the duration for the Render Engine thread to call the CPU to synthesize the target frame. At the same time, the initial threshold can be screened through the set threshold range to obtain a suitable first threshold.

[0009] In an alternative embodiment, monitoring the duration of the Render Engine thread calling the CPU to synthesize the target frame includes: SurfaceFlinger sending a first threshold to the monitoring sub-thread; the monitoring sub-thread monitoring the duration of the Render Engine thread calling the CPU to synthesize the target frame according to the first threshold. After calculating the appropriate first threshold, SurfaceFlinger can send the first threshold to the monitoring sub-thread. The monitoring sub-thread can monitor the duration of the Render Engine thread calling the CPU to synthesize the target frame according to the first threshold. In this way, the monitoring sub-thread can monitor the synthesis process of the target frame. When the duration of the Render Engine thread calling the CPU to synthesize the target frame exceeds the first threshold, it is convenient to adjust the working frequency of the CPU in a timely manner.

[0010] In an alternative embodiment, when the duration of the Render Engine thread calling the CPU to synthesize the target frame is greater than or equal to the first threshold, increasing the working frequency of the CPU to a specified frequency includes: when the duration of the Render Engine thread calling the CPU to synthesize the target frame is greater than or equal to the first threshold, the monitoring sub-thread increases the working frequency of the CPU from the initial working frequency to the specified frequency. In this way, the adjustment of the CPU working frequency can be realized by using the monitoring sub-thread.

[0011] In an alternative embodiment, after increasing the working frequency of the CPU to the specified frequency when the duration of the Render Engine thread calling the CPU to synthesize the target frame is greater than or equal to the first threshold, it further includes: when the Render Engine thread finishes calling the CPU to synthesize the target frame, the monitoring sub-thread adjusts the working frequency of the CPU from the specified frequency to the initial working frequency; wherein, the specified frequency is greater than the initial working frequency. When the duration of the Render Engine thread calling the CPU to synthesize the target frame is adjusted once, for the next frame after this target frame, it may not be necessary to adjust the CPU working frequency. Therefore, after the synthesis thread of the target frame ends, the working frequency of the CPU can be adjusted to the initial working frequency to reduce the power consumption of the CPU. In this way, dynamic frequency modulation of the CPU during the image refresh process can be achieved, and frequency increase can be realized when needed to reduce the possibility of frame loss, and the power consumption of the CPU can also be taken into account.

[0012] In an alternative implementation, the method for obtaining the specified frequency includes: acquiring the energy efficiency ratio curve of the CPU; calculating the slope between two adjacent points on the energy efficiency ratio curve; and determining the specified frequency based on the maximum value of the slope, where the specified frequency is less than the maximum operating frequency of the CPU. In this way, the specified frequency of the CPU can be selected according to the relationship between the operating frequency and power consumption of the CPU, so as to balance power consumption while increasing the operating frequency and achieve high-cost-effective frequency increase.

[0013] In an alternative implementation, determining the target historical frame according to the refresh rate of the current image frame includes: if the refresh rate of the current image frame is less than 90 Hz, taking the first historical frame before the current frame as the target historical frame; if the refresh rate of the current image frame is greater than or equal to 90 Hz, taking the second historical frame before the current frame as the target historical frame. In this way, a historical frame that has a greater impact on the current frame can be selected according to the refresh rate, and the operating frequency of the CPU can be reasonably adjusted.

[0014] In an alternative implementation, the method for calculating the initial threshold includes: multiplying the duration of the Render Engine thread calling the CPU to synthesize the target historical frame by an adjustment coefficient; where the adjustment coefficient is less than 1 and there is at least one adjustment coefficient. In this way, the obtained initial threshold is a threshold less than the duration of the Render Engine thread calling the CPU to synthesize the target historical frame.

[0015] In a second aspect, the present application provides an electronic device, which includes a display screen, a memory, and one or more processors; the display screen, the memory are coupled to the processor; wherein, the memory stores computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the dynamic frequency increase method of the CPU provided in the first aspect and any of its alternative implementations.

[0016] In a third aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions run on an electronic device, the electronic device executes the dynamic frequency increase method of the CPU provided in the first aspect and any of its alternative implementations.

[0017] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the dynamic frequency increase method of the CPU provided in the first aspect and any of its alternative implementations.

[0018] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fourth aspects above can refer to the beneficial effects in the first aspect and any possible design thereof, which will not be elaborated here. Brief Description of the Drawings

[0019] To more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic diagram of the screen after opening the photo album application provided in this embodiment;

[0021] Figure 2 is a schematic diagram of the screen after clicking on any picture in the photo album provided in this embodiment;

[0022] Figure 3 is a schematic diagram of the screen after clicking on the share operation in Figure 2 provided in this embodiment;

[0023] Figure 4 is a process diagram of the normal sliding of the share panel provided in this embodiment;

[0024] Figure 5 is a process diagram of the sliding of the share panel in the case of lag provided in this embodiment;

[0025] Figure 6 is provided in this embodiment Figure 3 layer decomposition diagram;

[0026] Figure 7 is a flow chart of a SurfaceFlinger composition thread provided in this embodiment;

[0027] Figure 8 is a schematic diagram of the structure of an electronic device provided in this embodiment;

[0028] Figure 9 is a schematic diagram of the hierarchical architecture of the software system of the electronic device provided in this embodiment;

[0029] Figure 10 is an architecture diagram of the display screen update process provided in this embodiment;

[0030] Figure 11 is a flow chart of a dynamic frequency boosting method of a CPU provided in this embodiment;

[0031] Figure 12 is an update process diagram of the normal situation of the image frame when the refresh rate is 60Hz provided in this embodiment;

[0032] Figure 13 is an update process diagram of the frame loss situation of the image frame when the refresh rate is 60Hz provided in this embodiment;

[0033] Figure 14 It is a process update diagram of the normal situation of an image frame when the refresh rate provided in this embodiment is 90Hz;

[0034] Figure 15 It is a process update diagram of the frame loss situation of an image frame when the refresh rate provided in this embodiment is 90Hz;

[0035] Figure 16 It is an energy efficiency ratio curve diagram of a CPU provided in this embodiment;

[0036] Figure 17 It is a process update diagram of an image frame after applying the dynamic frequency boosting method of the application CPU when the refresh rate provided in this embodiment is 60Hz;

[0037] Figure 18 It is a process update diagram of an image frame after applying the dynamic frequency boosting method of the application CPU when the refresh rate provided in this embodiment is 90Hz;

[0038] Figure 19 It is a flowchart of another dynamic frequency boosting method of a CPU provided in this embodiment;

[0039] Figure 20 It is a flowchart of yet another dynamic frequency boosting method of a CPU provided in this embodiment;

[0040] Figure 21 It is a schematic diagram of a chip structure provided in this embodiment. Specific embodiments

[0041] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0042] Hereinafter, terms such as "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "plurality" is two or more.

[0043] In addition, in this application, orientation terms such as "up", "down", "inside", and "outside" are defined relative to the orientation of the components shown in the drawings. It should be understood that these directional terms are relative concepts, which are used for relative description and clarification and can change accordingly with the change of the orientation of the components placed in the drawings.

[0044] For the convenience of subsequent understanding of the solution, the following terms are first explained:

[0045] Frame: When referring to the display screen of an electronic device, it is the smallest unit of a single frame of the picture. A frame can be understood as a still picture, and quickly and continuously displaying multiple connected frames can create the illusion of object movement. Before an electronic device displays a frame of the picture, the frame needs to be drawn, that is, frame drawing. After the frame drawing is completed, the electronic device can display the drawn frame.

[0046] Refresh rate: It refers to the number of times the display redraws the image per unit time, usually measured in Hertz (Hz). In the fields of computers and televisions, the refresh rate is used to describe the smoothness of the screen display image, usually how many times it is refreshed per second. Common refresh rates include 30Hz, 60Hz, 90Hz, and 120Hz, etc.

[0047] Dynamic effect: It refers to the effect of dynamic changes in the picture, and can also be called a dynamic effect or an animation effect, etc. The essence of the dynamic effect is the picture change effect brought about by the update of image frames (or it can be considered as the display of different image frames). Therefore, any picture change effect brought about by the update of image frames belongs to the scope of the dynamic effect of the embodiments of this application. Based on this, the embodiments of this application do not limit the specific types of dynamic effects too much.

[0048] Dynamic effects include but are not limited to the following possible dynamic effects: the dynamic effect when entering the multitasking interface from the home page through gesture navigation; the dynamic effect when entering the multitasking interface from the application interface using gesture navigation; the dynamic effect when entering the multitasking interface from the home page using three-button navigation; the dynamic effect when entering the multitasking interface from the application interface using three-button navigation; the dynamic effect when returning to the home page from the application interface using a swipe-up gesture; the dynamic effect when returning to the home page from the persistent card on the negative first screen using a swipe-up gesture; the dynamic effect when returning to the home page from a widget using a swipe-up gesture; the dynamic effect when opening an application from the desktop; the dynamic effect when entering the application page from the multitasking interface; the dynamic effect of desktop sliding (such as sliding from the home page to the negative first screen); the dynamic effect of unlocking to the desktop without a password; the dynamic effect of unlocking to the desktop with a password; the dynamic effect of pulling down and collapsing the notification bar, etc.

[0049] Vertical synchronization signal (Vsync): When the display screen finishes displaying the pixels of a row, it is called a row scan. When the display screen finishes displaying the pixels of all rows, it is called a field scan. The physical signal for the display screen to start scanning a field is called the vertical synchronization signal. After the display screen finishes scanning a field, it can display a frame of image data. That is, within one cycle of a Vsync signal, the display screen can display a frame of image data. How many fields the display screen can refresh per second, that is, how many frames of image data the display screen can display per second, is how many Vsync signals the electronic device needs to generate per second.

[0050] The Vsync signal can be generated by the Hardware Composer (HWC) of the electronic device. After the HWC generates the Vsync signal, the application can send the image to be displayed to the display screen within the cycle of this Vsync signal, so that the display screen can display this image data. That is to say, within the cycle corresponding to one Vsync signal, the module for the application to combine with the electronic device to perform image frame receipt can draw a frame of image data and send this frame of image data to the display screen, so that the display screen can display this frame of image data.

[0051] UI Thread: It refers to the main thread used to handle user interface operations and rendering. In an Android application, the UI thread is responsible for tasks such as handling user interactions, updating UI elements, and responding to events.

[0052] Surface is an abstract class that represents a target for drawing images. The application can submit its own image data to the SurfaceFlinger for processing and display through the interfaces provided by Surface. Surface can be held by multiple SurfaceViews, TextureViews, or other drawable views for displaying relevant content.

[0053] SurfaceTexture is a class used for texture rendering in the OpenGL ES environment. It inherits from TextureView and implements the SurfaceTexture.OnFrameAvailableListener interface at the same time. SurfaceTexture can be used as a texture target to receive streams from data sources such as the Camera and MediaPlayer and convert them into OpenGL ES textures for rendering. When a new video frame is available, SurfaceTexture will call the registered OnFrameAvailableListener callback function to notify the application.

[0054] OpenGL ES (OpenGL for Embedded Systems) is an OpenGL standard specifically designed for embedded systems. It is a subset of OpenGL and is mainly used for graphics rendering in resource-constrained environments such as mobile devices, game consoles, and embedded devices. OpenGL ES can use either hardware acceleration or software implementation and has the advantages of cross-platform, high performance, and scalability.

[0055] A Render Engine is a core component in computer graphics that is responsible for converting graphic data into the final visual result. It is a software or hardware module that processes and executes various graphic operations during the graphic rendering process. Common render engines include OpenGL, DirectX, Vulkan, etc.

[0056] The display screen of an electronic device is used to display images, and the displayed images can be updated under specified conditions. For example, when the display screen is a non-touch screen, the refresh and replacement of the displayed images can be controlled through a remote control, buttons, etc.; when the display screen is a touch screen, the display screen can receive user input events (such as pressing, swiping, etc.) and switch the displayed images accordingly. Or, the displayed images on the display screen can also be updated according to pop-up messages within the system.

[0057] When the displayed images on the display screen are updated, it is actually a process of continuously refreshing the image frames displayed on the display screen. Different image frame refresh rates can bring different visual experiences to users. A higher refresh rate can make the switching of the images displayed on the display screen smoother.

[0058] However, during the process of refreshing the image frames, there will occasionally be a stuttering phenomenon. Taking an electronic device with an Android system as an example:

[0059] When swiping up and down on the sharing panel, the sharing panel will experience stuttering and incoherence. When swiping left and right on the icon page of the shared link, there will also be stuttering. Especially when there is a blurred background on the sharing panel or the interface of the shared link, the stuttering is more obvious. When clicking on the chat dialog box in the WeChat application, the keyboard that should pop up together will experience stuttering and cannot pop up synchronously. It is even possible that when shutting down the device, the shutdown animation will experience stuttering.

[0060] It should be noted that the above sharing panel can be the panel when forwarding or sharing relevant information to other chat boxes or applications during WeChat chatting; or, it can be the panel displayed after clicking the share operation on the currently browsed content when browsing a certain application (such as Douyin, photo album, etc.). In this embodiment, the specific form of the sharing panel is not limited.

[0061] Figure 1 It is a schematic diagram of the screen after opening the photo album application provided by this embodiment.

[0062] Figure 2 It is a schematic diagram of the screen after clicking on any one of the pictures in the photo album provided by this embodiment.

[0063] Figure 3 It is a click provided by this embodiment Figure 2 The screen schematic diagram after the sharing operation in.

[0064] Exemplarily, taking the case where the user needs to perform a picture sharing operation as an example:

[0065] Combined with Figures 1 to 3 As shown, when the user opens the photo album application, the display screen of the electronic device will display multiple pictures (such as Figure 1 shown). The user can click on any one of the pictures, and the picture will be displayed full screen (such as Figure 2 shown). At the same time, multiple operations such as "Share", "Favorite", "Edit" and "More" will also be displayed below the picture. It can be understood that Figure 2 The operations in are only one example. In other solutions, other operations can also be displayed. When the user needs to share the picture, the user can click the "Share" operation, and the display screen will change from Figure 2 Switch to Figure 3 . Figure 3 The sharing panel of is located above the picture of Figure 2 . In the sharing panel, it can include Application 1, Application 2..., and these applications can be WeChat, Douyin, Xiaohongshu, etc. The applications in the sharing panel are not limited in this embodiment.

[0066] When there are many applications in the sharing panel, the user needs to operate the slider on the side of the sharing panel to slide the sharing panel to find the application needed for sharing.

[0067] Specifically, taking the case where the sharing panel slides up and down with jamming as an example, the cause of the jamming is analyzed.

[0068] Figure 4 It is a sliding process diagram of the sharing panel under normal circumstances provided by this embodiment.

[0069] Figure 5 It is a sliding process diagram of the sharing panel under the condition of jamming provided by this embodiment.

[0070] Combined with Figure 4 And Figure 5As shown, Frame 1, Frame 2, and Frame 3 are respectively an image frame. When the user swipes on the sharing panel, under normal circumstances, the display screen will display Frame 1 at time t1, the displayed screen on the display will be refreshed from Frame 1 to Frame 2 at time t2, and refreshed from Frame 2 to Frame 3 at time t3. It can be seen that during the normal refresh process, three image frames should be displayed within the time period from t1 to t4. When a stuttering phenomenon occurs, the displayed screen on the display will be refreshed from Frame 1 to Frame 2 at time t2, but it is not refreshed from Frame 2 to Frame 3 at time t3, resulting in a longer display time of Frame 2 and Frame 3 not being displayed normally. It can be seen that during the occurrence of stuttering, only two image frames are displayed within the time period from t1 to t4. In other words, when the displayed screen stutters, the image frames that should be displayed within the specified time are not displayed normally, and frame loss occurs, thus enabling the user to observe the stuttering phenomenon and affecting the user's usage experience.

[0071] Figure 6 is provided by this embodiment Figure 3 layer decomposition diagram.

[0072] As Figure 6 shown, the display screen of the electronic device with a sharing panel can include a background layer 10 and a sharing interface layer 20. Among them, the background layer 10 can include a picture layer 11 of the photo album and a status bar layer 12 above the screen. The sharing interface layer 20 can further include a sharing interface background layer 21, a sharing interface application layer 22, and a sharing interface interaction layer 23. Among them, the sharing interface application layer 22 is the layer where each application to be selected is located, and there is a corresponding relationship between the sharing interface application layer 22 and the sharing interface interaction layer 23. In order to improve the visual effect of the sharing interface and enhance the user's usage experience, the sharing interface background layer 21 can be set with a blurred background or a textured background, etc.

[0073] When the user triggers a sharing operation, within one cycle corresponding to a Vsync signal, the module of the sharing application that combines with the electronic device to draw image frames can draw and render the above-mentioned each layer, and the module that synthesizes image frames in the electronic device synthesizes each layer, so that multiple layers are combined into one image frame and sent to the buffer area. After the display driver waits for the Vsync signal to extract and display, the image frame is displayed on the display screen.

[0074] When the user slides on the sharing panel, the Vsync signal will continuously trigger the drawing, rendering and synthesis threads, thereby displaying one frame after another to achieve a coherent switching of the display screen. When frame loss occurs, it means that the drawing, rendering and synthesis threads are not completed normally within a Vsync signal cycle, resulting in the image frame to be refreshed not being cached to the cache area at the specified time. When the Vsync signal instructs the display driver to extract the display, it is unable to extract the next image frame normally, so that this frame cannot be updated, resulting in frame loss.

[0075] When drawing, rendering, and synthesizing image frames, the drawing and rendering threads are usually completed by the UI thread of the application. The application can store the image data drawn by the UI thread in the memory of the front buffer. The application can then submit the image data in the front buffer to the SurfaceFlinger service through Surface or SurfaceTexture. SurfaceFlinger synthesizes and blends the image data on all submitted Surfaces. It will synthesize the images of each Surface into a frame of image according to the view hierarchy, and blend effects such as transparency and masking. After the synthesis is completed, SurfaceFlinger stores the generated frame image in a memory called the back buffer. SurfaceFlinger sends the image data of the back buffer to the display (screen) and displays the image on the screen through the hardware scanning mechanism. In this way, the user can see the latest image content.

[0076] In the Android system, the UI thread of the application can currently complete the drawing and rendering work within the specified time, and store the relevant image data in the front buffer in a timely manner. The main source of frame loss is that the synthesis thread cannot complete the synthesis work on time.

[0077] Specifically, as users have higher requirements for the visual effects of the displayed screen, more material effects will be added when designing the image frame to improve the visual effect of the screen. In some electronic devices, the background layer of the sharing interface of the sharing panel is just a layer with a pure white background. In order to improve the visual effect of the screen, such as Figure 6 As shown, special effects such as blur and texture can be set in the sharing interface background layer 21 to make the synthesized picture effect better. However, for the synthesis thread, the synthesis time of the layer with a pure white background and other layers is shorter than the synthesis time of the layer with special effects and other layers. In other words, while improving the visual effect of the displayed picture, it is inevitable to increase the task of the synthesis thread and increase the duration of the synthesis thread. At this time, the image frame to be refreshed may not be completed and displayed within the specified time, resulting in frame loss.

[0078] Furthermore, when SurfaceFlinger performs the composition thread, SurfaceFlinger can call the RenderEngine to complete the rendering work in the composition stage. It can be understood that the drawing and rendering threads started in the application are mainly responsible for processing the graphics rendering tasks inside the application, including generating and processing OpenGL ES requests, etc. The rendering thread started in the SurfaceFlinger process is mainly responsible for rendering the data in the buffer onto the screen in a specific format to achieve the display effect of the image, including using the Render Engine for rendering and composition work.

[0079] Figure 7 It is a flowchart of a SurfaceFlinger composition thread provided by this embodiment.

[0080] As Figure 7 shown, when SurfaceFlinger performs the composition thread, it mainly includes calling the Render Engine located on the Central Processing Unit (CPU) for the first-stage composition. After the work in the CPU is completed, the preliminarily composed image will be submitted to the Graphics Processing Unit (GPU) for composition and finally cached in the Buffer. The working duration of the Render Engine will have a greater impact on the duration of the entire composition thread. In the Android system, a scheduling policy is set for the GPU to facilitate adjusting the duration of the threads running on the GPU.

[0081] In an electronic device, in order to achieve the compatibility of performance and power consumption, the processor in the electronic device will include multiple CPUs with different performances. Among these CPUs, some CPUs focus on performance and are used to execute tasks that require high computing power, such as games and video editing. Some CPUs focus on low power consumption and are used to execute low-load tasks. In this way, the electronic device can dynamically allocate tasks according to the requirements of the application, improving the overall performance and energy efficiency ratio of the system. For example, when the system load is light, the low-power CPU can work alone to ensure the long battery life of the system; while when the system requires higher computing power, the high-performance CPU can be activated to provide better response speed and execution efficiency.

[0082] Currently, the Render Engine invoked by SurfaceFlinger runs on a low-power CPU to reduce the power consumption of the electronic device. However, there are multiple low-power CPUs in the electronic device, and different CPUs have different scheduling policies. As a result, the CPU that causes frame drops does not have a corresponding scheduling policy set for the frame drop phenomenon, so the frame drop and lag situations cannot be changed, seriously affecting the user experience.

[0083] To solve the frame drop problem caused by the long running time of the composition thread in the electronic device, this application provides a dynamic frequency boosting method for the CPU to set a reasonable scheduling policy for the low-power CPU, reduce the possibility of frame drops, and take into account the power consumption of the CPU. This method can be applied to electronic devices.

[0084] It can be understood that the electronic device can be a mobile phone, a tablet computer, a handheld computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device, etc. The specific type of the electronic device in the embodiments of this application is not particularly limited.

[0085] Figure 8 It is a schematic structural diagram of an electronic device provided in this embodiment.

[0086] Such as Figure 8As shown in the figure, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 180, a key 190, a display screen 193, a subscriber identification module (SIM) card interface 194, and a camera 195, etc. Among them, the sensor module 180 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0087] The processor (Central Processing Unit, CPU) 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0088] The controller may be the nerve center and command center of the electronic device. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0089] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0090] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0091] The charging management module 140 is configured to receive a charging input from a power supply device (such as a charger, laptop power supply, etc.). Among them, the charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device.

[0092] While charging the battery 142, the charging management module 140 may also supply power to the electronic device through the power management module 141. Among them, the battery 142 may specifically be composed of multiple batteries connected in series. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.

[0093] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives the input from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the display screen 193, the camera 195, the wireless communication module 160, etc. The power management module 141 may also be used to monitor parameters such as the voltage, current, battery cycle count, and battery health status (leakage, impedance) of the battery. In some other embodiments, the power management module 141 may also be provided in the processor 110.

[0094] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external non-volatile memory.

[0095] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 110, and can be used to store the operating system or executable programs of other running programs (such as machine instructions), and can also be used to store user and application data, etc. The non-volatile memory can also store executable programs and store user and application data, etc., and can be pre-loaded into the random access memory for direct reading and writing by the processor 110.

[0096] The touch sensor, also known as the "touch control device". The touch sensor can be disposed on the display screen 193, and the touch sensor and the display screen 193 form a touch screen, also known as the "touch screen". The touch sensor is used to monitor touch operations acting on or near it. The touch sensor can transmit the monitored touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 193. In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device, at a different position from the display screen 193.

[0097] The ambient light sensor is used to sense the ambient light brightness. For example: The ambient light sensor can measure the light intensities of four channels of the ambient light. The ambient light sensor outputs the light intensities of the four channels of the measured ambient light to the processor 110. The processor 110 can process the light intensities of the four channels of the ambient light output by the ambient light sensor to obtain the light intensity of the ambient light. In the bright screen state, the electronic device can adaptively adjust the display screen brightness according to the obtained light intensity of the ambient light.

[0098] A pressure sensor is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor may be disposed on the display screen 193. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. When a touch operation acts on the display screen 193, the electronic device monitors the intensity of the touch operation according to the pressure sensor. The electronic device can also calculate the position of the touch according to the monitoring signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure duration threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure duration threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0099] In some embodiments, the electronic device may include one or N cameras 195, where N is a positive integer greater than 1. In the embodiments of the present application, the types of the cameras 195 can be distinguished according to the hardware configuration and the physical location. For example, the camera disposed on the side of the display screen 193 of the electronic device can be called a front camera, and the camera disposed on the back cover of the electronic device can be called a rear camera; for another example, a camera with a short focal length and a large viewing angle can be called a wide-angle camera, and a camera with a long focal length and a small viewing angle can be called a normal camera. Among them, the length of the focal length and the size of the viewing angle are relative concepts and there are no specific parameter limitations. Therefore, the wide-angle camera and the normal camera are also relative concepts and can be specifically distinguished according to physical parameters such as the focal length and the viewing angle.

[0100] The electronic device realizes the display function through the GPU, the display screen 193, and the application processor, etc. The GPU is a microprocessor for image editing, connecting the display screen 193 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0101] The electronic device can realize the shooting function through the ISP, the camera 195, the video codec, the GPU, the display screen 193, and the application processor, etc. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information. In the embodiments of the present application, in the frame drawing process of each image frame, the function of the GPU will be used to make the finally displayed picture obtain better display effects and performance.

[0102] The ISP is used to process the data fed back by the camera 195. For example, when taking a photo, the shutter is opened, and light passes through the lens and is transmitted to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted by the camera's photosensitive element to the ISP for processing and converted into an image visible to the naked eye. The ISP can also optimize the noise and brightness of the image through algorithms. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be provided in the camera 195. The camera 195 is used to capture still images or videos.

[0103] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0104] The display screen 193 is used to display images, videos, etc. The display screen 193 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N display screens 193, where N is a positive integer greater than 1.

[0105] In the embodiments of the present application, the display screen 193 can be used to display the pages required by the electronic device (such as a wizard page (including a highlight recommendation page and an external module access page), etc.), and display the images captured by any one or more cameras 195 on this interface.

[0106] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modem, and the baseband processor, etc.

[0107] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas.

[0108] The mobile communication module 150 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to an electronic device. The mobile communication module 150 may receive electromagnetic waves through antenna 1, filter, amplify, and perform other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be provided in the same device.

[0109] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 193. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and provided in the same device as the mobile communication module 150 or other functional modules.

[0110] The wireless communication module 160 may provide solutions for wireless communications applied to an electronic device, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves through antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 may also receive the signal to be transmitted from the processor 110, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through antenna 2 for radiation.

[0111] The SIM card interface 194 is used to connect to a SIM card. The SIM card can be inserted into or removed from the SIM card interface 194 to achieve contact and separation from the electronic device. The electronic device can support one or more SIM card interfaces. The SIM card interface 194 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 194 simultaneously. The SIM card interface 194 can also be compatible with external memory cards. The electronic device interacts with the network through the SIM card to implement functions such as calls and data communication. One SIM card corresponds to one user number.

[0112] It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are only illustrative descriptions and do not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0113] It can be understood that generally, in addition to the support of hardware, the implementation of the functions of an electronic device also requires the cooperation of software. The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, taking the layered architecture system as an example, the software structure of the electronic device is exemplarily described.

[0114] Figure 9 is a schematic diagram of the layered architecture of the software system of the electronic device provided in this embodiment.

[0115] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers is through software interfaces (such as APIs).

[0116] In some examples, as shown in Figure 9 In the embodiments of the present application, the software of the electronic device is divided into five layers, from top to bottom, namely the application layer (or called the application layer), the framework layer (or called the application framework layer), the system library and Android runtime, the HAL layer (hardware abstraction layer), and the driver layer (or called the kernel layer). Among them, the system library and Android runtime can also be called the native framework layer or the native layer.

[0117] The application package can include applications such as phone, email, calendar, camera, etc.

[0118] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0119] As Figure 9 shown, the application framework layer may include a window manager, a frame rate control system, an image composition system, a view system, a package manager, an input manager, an activity manager, and a resource manager, etc.

[0120] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0121] The frame rate control system is used to adjust the screen refresh rate.

[0122] The image composition system is used to control image composition and generate a vertical synchronization (Vsync) signal.

[0123] The image composition system includes: a composition thread, a Vsync thread, and a cache queue thread. The composition thread is used to be awakened by the Vsync signal for composition. The Vsync thread is used to request the generation of the next Vsync signal according to the Vsync signal. The cache queue thread is used to store caches, generate Vsync signal requests, and wake up the composition thread, etc.

[0124] The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying a picture.

[0125] The package manager is used for program management within the system, such as: application installation, uninstallation, and upgrade, etc.

[0126] The input manager is used for programs that manage input devices. For example, the input system can determine input operations such as mouse click operations, keyboard input operations, and touch swipes.

[0127] The activity manager is used to manage the life cycles of various applications and the navigation back function. It is responsible for creating the main thread of Android and maintaining the life cycles of various applications.

[0128] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0129] The Android runtime includes the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0130] The core libraries consist of two parts: one is the functional functions that need to be called by the Java language, and the other is the core libraries of Android.

[0131] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0132] The system libraries can include multiple functional modules. For example: image rendering library, image composition library, function library, media library, and input processing library, etc.

[0133] The image rendering library is used for the rendering of two-dimensional or three-dimensional images. The image composition library is used for the composition of two-dimensional or three-dimensional images.

[0134] In a possible implementation, the application renders the image through the image rendering library, and then the application sends the rendered image to the cache queue of the image composition system. Whenever the Vsync signal arrives, the image composition system (e.g., Surfaceflinger) sequentially obtains a frame of image to be composed from the cache queue, and then performs image composition through the image composition library.

[0135] The function library provides macros, type definitions, string manipulation functions, mathematical calculation functions, and input / output functions used in the C language, etc.

[0136] The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG, etc.

[0137] The input processing library is a library for processing input devices, and can implement mouse, keyboard, and touch input processing, etc.

[0138] The hardware abstraction layer can include multiple library modules. The library modules can be, for example, the hardware composer (hwcomposer, HWC), the camera library module, etc. The Android system can load the corresponding library modules for the device hardware, thereby achieving the purpose of the application framework layer accessing the device hardware. The device hardware can include, for example, the LCD display screen, the camera, etc. in the electronic device.

[0139] The kernel layer is the layer between hardware and software. The kernel layer at least includes touch panel (TP) driver, display driver, Bluetooth driver, WIFI driver, keyboard driver, shared memory driver, camera driver, etc.

[0140] The hardware can be audio device, Bluetooth device, camera device, sensor device, etc.

[0141] Figure 10 It is an architecture diagram of a display screen update process provided by this embodiment.

[0142] As Figure 10 shown, when the user touches the touch panel, the touch sensor in the touch panel will receive the touch operation. The touch sensor in the kernel layer processes the touch operation into an input event (including information such as touch coordinates, touch force, timestamp of the touch operation, etc.). The input event is stored in the kernel layer. The kernel layer can report the input event to the input manager in the application framework layer. The input manager locates the target application according to the input event.

[0143] The target application calls the image rendering library in the system library through the view system in the application framework layer to draw and render the image. The target application sends the drawn and rendered image to the cache queue of the image composition system. The image composition library in the system library composes the drawn and rendered images in the image composition system into the target interface. The image composition system enables the screen (display screen) to display the corresponding interface of the target application through the display driver in the kernel layer.

[0144] For example, when the user clicks on the photo album application in the display screen, the touch sensor processes the click operation into an input event, which includes the position, force, and time of the click, etc. The input manager locates the photo album application according to the position information contained in the input event and confirms the operation on the photo album application according to other information. For example, long pressing the photo album application is to move or delete the photo album application icon, and short pressing the photo album application is to open the photo album application.

[0145] The photo album application calls the image rendering library in the system library through the view system to draw and render the image. The photo album application sends the drawn and rendered image to the cache queue of the image composition system. The image composition library in the system library composes the drawn and rendered images in the image composition system into the photo album interface. The image composition system enables the screen (display screen) to display the corresponding interface of the photo album application through the display driver in the kernel layer.

[0146] Figure 11 It is a flowchart of a dynamic frequency boosting method for CPU provided by this embodiment.

[0147] As Figure 11As shown in the first aspect, this embodiment provides a method for dynamically boosting the frequency of a CPU, including:

[0148] Step S200: Detect whether a frame is dropped in the current frame.

[0149] Step S210: When a frame is dropped in the current frame, detect the refresh rate of the current image frame.

[0150] Among them, the current frame refers to the image frame that is being displayed or about to be displayed on the display screen. The update process of the current frame is a continuous loop and is carried out within the display cycle of each frame. By continuously updating the current frame, the Android system can provide smooth image display and animations to meet the user's requirements for visual experience.

[0151] Exemplarily, Surfaceflinger can automatically detect whether a frame is dropped in the current frame. When Surfaceflinger detects that a frame is dropped, when detecting the current refresh rate, it can use the interface of Surfaceflinger to create SurfaceSession and SurfaceControl objects for the default display object, such as Display, and then bind SurfaceControl and SurfaceSession, so as to be able to detect the current refresh rate.

[0152] It can be understood that the refresh rate of the current image frame can also be detected through a window manager, etc. In this embodiment, only one example is provided and is not limited.

[0153] Since the update situations of image frames corresponding to different refresh rates are different, and the influences of different historical frames on the current frame where the frame drop event occurs are also different. Therefore, before adjusting the working frequency of the CPU, it is first necessary to determine the refresh rate at which the current frame is located, so as to reasonably adjust the working frequency of the CPU, while reducing the risk of frame drops and reducing power consumption losses.

[0154] Among them, the historical frame refers to the image frame that was displayed before the current frame.

[0155] Step S220: Determine the target historical frame according to the refresh rate of the current image frame.

[0156] Since different refresh rates have different historical frames that have a greater impact on the current frame, it is necessary to determine the target historical frame according to the refresh rate in order to obtain a suitable duration threshold later.

[0157] Exemplarily, the above step S220 can be implemented through the following steps S221 - step S222.

[0158] Step S221: If the refresh rate of the current image frame is less than 90 Hz, the first historical frame before the current frame is used as the target historical frame.

[0159] When the image frame is updated, the image frame to be displayed after being synthesized by the synthesis thread is first stored in the Buffer. The display driver reads the cached image frame from the Buffer and replaces the image frame being displayed on the display screen to achieve the update of the image frame.

[0160] For the case where the refresh rate is less than 90 Hz, such as 60 Hz, 30 Hz, etc., the time of its own refresh cycle is relatively long, and one image frame can be cached in the Buffer to normally complete the refresh requirement of the image frame. The following takes 60 Hz as an example to illustrate the update process of the image frame.

[0161] Figure 12 It is the update process diagram of the normal situation of the image frame with a refresh rate of 60 Hz provided by this embodiment.

[0162] As Figure 12 shown, under normal circumstances, the synthesis thread of the image frame A includes two stages. Among them, the RenderEngine synthesis thread performs the first-stage synthesis on the image frame to form the image frame A', and the first-stage synthesis runs on the CPU. After the Render Engine thread calls the CPU to synthesize the image frame A', the CPU submits the image frame A' to the GPU for the second-stage synthesis. The image frame A' undergoes the second-stage synthesis to form the image frame A. Compared with the image frame A, the image frame A' is the intermediate state of the image frame A before the entire synthesis thread is completed. The image frame A formed through the synthesis thread is cached in the Buffer before the moment t1 to wait for the display driver to read. In this way, the display driver can normally read the image frame A from the Buffer at the moment t1 and perform the display in the 1- t2 time period.

[0163] During the display of image frame A, the composition thread of image frame B is synchronized. The composition thread of image frame B includes two stages. Among them, the Render Engine composition thread performs the first-stage composition on the image frame to form image frame B', and the first-stage composition runs on the CPU. After the Render Engine thread calls the CPU to synthesize image frame B', the CPU submits image frame B' to the GPU for the second-stage composition. Image frame B' undergoes the second-stage composition to form image frame B. Compared with image frame B, image frame B' is an intermediate state of image frame B before the completion of the entire composition thread. The image frame B formed through the composition thread is cached in the Buffer before time t2 to wait for the display driver to read. In this way, the display driver can normally read image frame B from the Buffer at time t2 and replace image frame A to achieve the display refresh from image frame A to image frame B, so as to 2- display image frame B during the time period from t to t3.

[0164] Figure 13 It is an update process diagram of the frame loss situation of image frames with a refresh rate of 60Hz provided in this embodiment.

[0165] As Figure 13 shown, the composition thread of image frame A includes two stages. Among them, the Render Engine composition thread performs the first-stage composition on the image frame to form image frame A', and the first-stage composition runs on the CPU. After the Render Engine thread calls the CPU to synthesize image frame A', the CPU submits image frame A' to the GPU for the second-stage composition. Image frame A' undergoes the second-stage composition to form image frame A. The image frame A formed through the composition thread is cached in the Buffer before time t1 to wait for the display driver to read. In this way, the display driver can normally read image frame A from the Buffer at time t1 and display it during the time period from t 1- to t2.

[0166] During the display of image frame A, the composition thread of synthesizing image frame B by the Render Engine thread calling the CPU is synchronized. The composition thread of image frame B includes two stages. Among them, the Render Engine composition thread performs the first-stage composition on the image frame to form image frame B', and the first-stage composition runs on the CPU. After the Render Engine thread calls the CPU to synthesize image frame B', the CPU submits image frame B' to the GPU for the second-stage composition. Image frame B' undergoes the second-stage composition to form image frame B. By comparing Figure 12 and Figure 13 it can be seen that Figure 13The duration of the synthesis of the image frame B in the first stage is relatively long, resulting in the failure to complete the synthesis of the image frame B in the GPU during the second stage before the time t2. At this time, the image frame B formed by the synthesis thread is cached in the Buffer after the time t2. At the time t2, when the display driver reads the image frame B from the Buffer, since the image frame B has not been synthesized at the time t2 and cannot be cached in the Buffer, the image frame B cannot be read normally, and the display screen cannot display the image frame B during the time period from t 2- During the time period from t to t3, the image frame B cannot be displayed, and the image frame A is still displayed. It can be seen that at the moment when the image frame B should be displayed, it cannot be displayed normally, resulting in the loss of the image frame B.

[0167] It should be noted that when the refresh rate is 30Hz, the refresh process of the display screen is similar to that of 60Hz, which will not be elaborated here.

[0168] According to the analysis when the above-mentioned frame loss phenomenon occurs, for a refresh rate less than 90Hz, the historical frame that has a greater impact on the frame loss phenomenon is the synthesis thread of the first frame historical frame before the current frame. During the synthesis cycle of the previous frame, if the target historical frame cannot be synthesized within the specified time, then the frame loss phenomenon will occur in the current frame. In this way, when the refresh rate is less than 90Hz, it is more valuable to select the first frame historical frame before the current frame as the target historical frame, and it is more capable of reasonably adjusting the working frequency of the CPU.

[0169] Step S222: If the refresh rate of the current image frame is greater than or equal to 90Hz, then use the second frame historical frame before the current frame as the target historical frame.

[0170] For a refresh rate greater than or equal to 90Hz, such as 90Hz, 120Hz, etc., the time of its own refresh cycle is relatively short, and two image frames can be cached in the Buffer to normally meet the refresh requirements of the image frames. The following takes 90Hz as an example to illustrate the update process of the image frames.

[0171] Figure 14 It is the update process diagram of the normal situation of the image frames when the refresh rate is 90Hz provided in this embodiment.

[0172] Such as Figure 14As shown, under normal circumstances, the synthesis thread of image frame A includes two stages. Among them, the RenderEngine synthesis thread performs the first-stage synthesis on the image frame to form image frame A', and the first-stage synthesis runs on the CPU. After the Render Engine thread calls the CPU to synthesize image frame A', the CPU submits image frame A' to the GPU for the second-stage synthesis. Image frame A' undergoes the second-stage synthesis to form image frame A. Compared with image frame A, image frame A' is an intermediate state of image frame A before the entire synthesis thread is completed. The image frame A formed through the synthesis thread is cached in the Buffer before time t1 to wait for the display driver to read it.

[0173] After image frame A is cached in the Buffer, the synthesis of image frame B starts. The synthesis thread of image frame B includes two stages. Among them, the Render Engine synthesis thread performs the first-stage synthesis on the image frame to form image frame B', and the first-stage synthesis runs on the CPU. After the Render Engine thread calls the CPU to synthesize image frame B', the CPU submits image frame B' to the GPU for the second-stage synthesis. Image frame B' undergoes the second-stage synthesis to form image frame B. Compared with image frame B, image frame B' is an intermediate state of image frame B before the entire synthesis thread is completed. The image frame B formed through the synthesis thread is cached in the Buffer before time t2 to wait for the display driver to read it. In this way, the display driver can normally read image frame A from the Buffer at time t2 and display it in the 2- time period t3.

[0174] In the case of a refresh rate of 90Hz, when the display driver reads the image frames to be updated from the Buffer, two image frames are cached in the Buffer to facilitate smooth refreshing at a refresh rate of 90Hz. When multiple image frames are cached in the Buffer, the display driver reads the image frames cached in the Buffer according to the first-in, first-out principle to achieve sequential refreshing.

[0175] While Image Frame A is being displayed, the composition thread of Image Frame C is running synchronously. The composition thread of Image Frame C includes two stages. Among them, the Render Engine composition thread performs the first-stage composition on the image frame to form Image Frame C’, and the first-stage composition runs on the CPU. After the Render Engine thread calls the CPU to synthesize Image Frame C’, the CPU submits Image Frame C’ to the GPU for the second-stage composition. Image Frame C’ undergoes the second-stage composition to form Image Frame C. Compared with Image Frame C, Image Frame C’ is an intermediate state of Image Frame C before the entire composition thread is completed. The Image Frame C formed through the composition thread is cached in the Buffer before time t3 to wait for the display driver to read. At this time, Image Frame B and Image Frame C are cached in the Buffer. The display driver reads Image Frame B from the Buffer at time t3 and replaces Image Frame A to achieve the display refresh from Image Frame A to Image Frame B, so as to display Image Frame B in the time period from t 3- to t4. At this time, there will still be one remaining frame of Image Frame C in the Buffer to be read at time t4 and displayed in the time period from t 4- to t5.

[0176] It can be understood that during the display of Image Frame B or Image Frame C, other image frames will also be synthesized synchronously, which are not shown one by one in the figure.

[0177] Figure 15 is the update process diagram of the frame loss situation of the image frames when the refresh rate provided in this embodiment is 90Hz.

[0178] As Figure 15 shown, the composition thread of Image Frame A includes two stages. Among them, the Render Engine composition thread performs the first-stage composition on the image frame to form Image Frame A’, and the first-stage composition runs on the CPU. After the RenderEngine thread calls the CPU to synthesize Image Frame A’, the CPU submits Image Frame A’ to the GPU for the second-stage composition. Image Frame A’ undergoes the second-stage composition to form Image Frame A. Compared with Image Frame A, Image Frame A’ is an intermediate state of Image Frame A before the entire composition thread is completed. The Image Frame A formed through the composition thread is cached in the Buffer before time t1 to wait for the display driver to read.

[0179] After the image frame A is cached in the Buffer, the synthesis of the image frame B starts. The synthesis thread of the image frame B includes two stages. Among them, the Render Engine synthesis thread performs the first-stage synthesis on the image frame to form the image frame B', and the first-stage synthesis runs on the CPU. After the Render Engine thread calls the CPU to synthesize the image frame B', the CPU submits the image frame B' to the GPU for the second-stage synthesis, and the image frame B' forms the image frame B after the second-stage synthesis.

[0180] By comparing Figure 14 and Figure 15 it can be seen that in the synthesis thread of the image frame B, the duration of the first-stage synthesis to form the image frame B' is relatively long, resulting in the image frame B not being normally cached in the Buffer before the moment t3. At the moment t2, after the image frame A is read and displayed, there is no other image frame to be refreshed cached in the Buffer. It can be seen that in the time period from t3 to t4, the image frame A is normally displayed, while the image frame B is not synthesized and cached during this period. In this way, at the moment t3, when the display driver needs to read the next image frame B, it cannot be obtained. At this time, the image frame A continues to be displayed in the time period from t3 to t4, and the image frame B is missing, resulting in the frame loss phenomenon of the image frame B in the time period from t3 to t4. And the image frame B is cached in the Buffer in the time period from t3 to t4. When the display driver reads the image frame to be updated from the Buffer at the moment t4, it can read the image frame B and replace the image frame A with it.

[0181] It should be noted that when the refresh rate is 120Hz, the refresh process of the display screen is similar to that of 90Hz, which will not be elaborated here.

[0182] According to the analysis when the above frame loss phenomenon occurs, it can be seen that when the frame loss phenomenon occurs, the image frame B that should have been displayed in the time period from t3 to t4 should have been synthesized and completed in the time period from t1 to t2. It can be seen that for the current frame with the frame loss phenomenon, the image frame that has a greater impact on it is the second historical frame before the current frame. In other words, for a refresh rate greater than or equal to 90Hz, the target historical frame that has a greater impact on the frame loss phenomenon is the second historical frame before the current frame. During the synthesis cycle of the target historical frame, if the target historical frame cannot be synthesized within the specified time, then the frame loss phenomenon will occur in the current frame. In this way, when the refresh rate is greater than or equal to 90Hz, it is more valuable to select the second historical frame before the current frame as the target historical frame, and it is more able to reasonably adjust the working frequency of the CPU.

[0183] Exemplarily, when SurfaceFlinger is running, a selection comparison table of the refresh rate and the target historical frames can be stored in the composition system. When SurfaceFlinger detects the refresh rate of the current frame, the target historical frames can be selected according to this comparison table for subsequent calculations.

[0184] Step S230: Calculate an initial threshold based on the duration of the Render Engine thread calling the CPU to compose the target historical frames.

[0185] Among them, the initial threshold is a duration threshold.

[0186] During the running of SurfaceFlinger, SurfaceFlinger can record the running time of the entire composition thread and the running time of each stage. After determining the target historical frames, SurfaceFlinger can extract the duration of the Render Engine thread calling the CPU to compose the target historical frames recorded by it, and calculate the initial threshold according to this duration, so as to obtain an adjustment benchmark for the duration of the Render Engine thread calling the CPU to compose the target frames after the current frame by using the initial threshold.

[0187] Among them, the target frame refers to the image frame being currently composed, and this image frame can be refreshed after the current frame.

[0188] Exemplarily, the calculation method of the initial threshold can perform a multiplication calculation on the duration of the Render Engine thread calling the CPU to compose the target historical frames and an adjustment coefficient. Among them, the adjustment coefficient is less than 1. In this way, the obtained initial threshold is a duration threshold less than the duration of the Render Engine thread in the historical frames.

[0189] For example, if the duration of the Render Engine thread calling the CPU to compose the target historical frames is 3 ms, 0.7 can be selected as the adjustment coefficient, and the initial threshold obtained is 2.1 ms; or, if the duration of the Render Engine thread calling the CPU to compose the target historical frames is 2 ms, 0.875 can be selected as the adjustment coefficient, and the initial threshold obtained is 1.75 ms.

[0190] It should be noted that if the initial threshold is obtained through the above calculation method, different calculation adjustment coefficients can be selected in different calculation cycles (for different target frames). Or rather, multiple adjustment coefficients can be provided during the calculation to facilitate the selection of appropriate adjustment coefficients for calculation.

[0191] When there are multiple adjustment coefficients, a corresponding relationship between the duration of the Render Engine thread calling the CPU to synthesize the target historical frame and the adjustment coefficient can be established. This corresponding relationship can be a direct proportional relationship, an inverse proportional relationship, or a non-linear corresponding relationship, etc. In this way, the corresponding adjustment coefficient can be obtained according to the duration of the Render Engine thread calling the CPU to synthesize different target historical frames, so as to obtain a suitable initial threshold.

[0192] Exemplarily, if the duration of the Render Engine thread calling the CPU to synthesize the target historical frame is greater than or equal to 2 ms and less than 3 ms, at this time, the adjustment coefficient 0.7 can be corresponding. If the duration of the Render Engine thread calling the CPU to synthesize the target historical frame is greater than or equal to 1 ms and less than 2 ms, at this time, the adjustment coefficient 0.8 can be corresponding.

[0193] It can be understood that the corresponding relationship between the duration of the Render Engine thread calling the CPU to synthesize the target historical frame and the adjustment coefficient provided in this embodiment is only an example, and the specific corresponding relationship is not limited in this embodiment.

[0194] Step S240: Determine whether the initial threshold is within the set threshold range.

[0195] The initial threshold is limited by the set threshold range to prevent incorrect values such as 0, positive infinity, and negative numbers that cannot be achieved due to calculation errors when calculating the initial threshold.

[0196] Or, since the duration of the Render Engine thread calling the CPU to synthesize the target historical frame is too long and the selected adjustment coefficient is large, the obtained initial threshold is still timeout, and such an adjustment is also unreasonable. For example, if the duration of the RenderEngine thread calling the CPU to synthesize the target historical frame is 3 ms and the selected adjustment coefficient is 0.9, the obtained initial threshold is 2.7 ms at this time. And a duration less than 2.2 ms can make the target frame synthesized within the specified time, so the obtained initial threshold does not meet the adjustment requirements at this time.

[0197] Exemplarily, the set threshold range can be obtained based on actual operation experience.

[0198] Alternatively, setting a threshold range can also use the duration of the CPU synthesis called by the RenderEngine thread for the image frames without frame loss events before the current frame as a reference. For example, when there are no frame loss events in the five consecutive frames before the current frame, the first three consecutive frames among these five frames can be taken as reference frames, and the maximum value of the duration of the CPU synthesis called by the Render Engine thread in these three reference frames is used as the upper limit of the set threshold range, and the minimum value of the duration of the CPU synthesis called by the Render Engine thread in these three reference frames is used as the lower limit of the set threshold range, so as to obtain a reasonable set threshold range.

[0199] Step S251: If the initial threshold meets the set threshold range, use the initial threshold as the first threshold.

[0200] Among them, the first threshold is a duration threshold.

[0201] In this way, the initial threshold can be screened through the set threshold range to obtain a suitable first threshold.

[0202] Step S252: If the initial threshold does not meet the set threshold range, repeat the above steps S210 - S240.

[0203] In this way, when a suitable first threshold is not obtained during the first frame loss phenomenon, the working frequency of the RenderEngine thread calling the CPU to synthesize the target frame can be temporarily not adjusted. When the frame loss phenomenon occurs again next time, the above steps can be repeated. After obtaining a suitable first threshold, the working frequency of the Render Engine thread calling the CPU to synthesize the target frame is adjusted, so as to prevent a large impact on the power consumption of the CPU.

[0204] Step S260: Surfaceflinger sends the first threshold to the monitoring sub - thread.

[0205] When a suitable first threshold is calculated, Surfaceflinger can send the first threshold to the monitoring sub - thread.

[0206] Among them, the monitoring sub - thread is a sub - thread in the synthesis thread used to monitor the duration of the Render Engine thread calling the CPU to synthesize the target frame.

[0207] Step S270: The monitoring sub - thread monitors the duration of the Render Engine thread calling the CPU to synthesize the target frame according to the first threshold.

[0208] The monitoring sub-thread can monitor the duration of the Render Engine thread calling the CPU to synthesize the target frame according to the first threshold. In this way, the monitoring sub-thread can monitor the target frame. When the duration of the Render Engine thread calling the CPU to synthesize the target frame exceeds the first threshold, it is convenient to timely adjust the working frequency of the CPU.

[0209] Step S280: When the monitoring sub-thread monitors that the duration of the Render Engine thread for the target frame is greater than or equal to the first threshold, the monitoring sub-thread increases the working frequency of the CPU from the initial working frequency to the specified frequency.

[0210] Among them, the initial working frequency refers to the running working frequency of the CPU before being adjusted to the specified frequency. The specified frequency can be a frequency set in the system, and the initial working frequency is less than the specified frequency. When the monitoring sub-thread needs to increase the working frequency of the CPU, it can automatically increase the working frequency of the CPU from the initial working frequency to the specified frequency.

[0211] When the duration of the Render Engine thread calling the CPU to synthesize the target frame is greater than or equal to the first threshold, the duration of the entire synthesis thread of the target frame may be too long. At this time, the working frequency of the CPU can be adjusted in time to shorten the running time of the Render Engine thread calling the CPU, thereby reducing the possibility of the duration of the target frame synthesis thread timing out and reducing the possibility of frame loss occurring again, so that the display screen can be smoother.

[0212] In some embodiments, the specified frequency can be obtained in the following manner:

[0213] Step S281: Obtain the energy efficiency ratio curve of the CPU.

[0214] Figure 16 is an energy efficiency ratio curve graph of a CPU provided in this embodiment.

[0215] As Figure 16 shown, in the energy efficiency ratio curve of the CPU, the abscissa is the performance of the CPU, and the ordinate is the power consumption of the CPU. Among them, the performance refers to the computing power of the CPU (DMIPS: Dhrystone Million Instructions executed Per Second), which is used to measure the number of Dhrystone million instructions executed per second by a computer or processor. The energy efficiency ratio curve of the CPU generally refers to the relationship curve between performance and power consumption, which shows how the performance of the CPU changes with power consumption under different workloads. Generally speaking, the steeper the energy efficiency ratio curve, the higher the performance obtained while maintaining a lower power consumption, which means that the processor achieves a better balance between performance and power consumption.

[0216] Since there is a corresponding relationship between the performance and the operating frequency of the CPU, the energy efficiency ratio curve of the CPU can reflect the relationship between the operating frequency and the power consumption while reflecting the relationship between the performance and the power consumption. In this way, it is possible to select a specified frequency of the CPU according to the relationship between the performance and the power consumption of the CPU, so as to take into account the power consumption while increasing the operating frequency and achieve high-cost performance frequency increase.

[0217] It can be understood that different CPUs have different energy efficiency ratio curves. Therefore, when selecting the specified frequency, it is necessary to select according to the energy efficiency ratio curve of the CPU to obtain a more appropriate specified frequency for the CPU.

[0218] Step S282: Calculate the slope between two adjacent points on the energy efficiency ratio curve.

[0219] Since the slope can reflect the growth rate of the power consumption as the performance increases. In this way, by calculating the slope, it is possible to obtain the increase in power consumption when the performance of the CPU increases. In other words, by calculating the slope, it is possible to obtain the increase in power consumption when the operating frequency of the CPU increases.

[0220] Step S283: Determine the specified frequency based on the maximum value of the slope, and the specified frequency is less than the maximum operating frequency of the CPU.

[0221] If there is a maximum value for the slope between two points, it means that when the performance of the CPU changes from the smaller performance to the larger performance between these two points, the power consumption increases the most. It can be seen that if there is a maximum value for the slope between two points, it means that when the operating frequency of the CPU changes from the smaller operating frequency to the larger operating frequency between these two points, the power consumption increases the most. Selecting the smaller operating frequency between these two points as the specified frequency can avoid excessive increase in the power consumption of the CPU and reduce the operating load of the CPU and the electronic device.

[0222] As Figure 16 shown, in Figure 16 , let the slope between points M and N be k1, and let the slope between points N and P be k2. As can be seen from the figure, k2 is greater than k1, and k2 is the maximum value of the slope between adjacent points in the entire energy efficiency ratio curve. At this time, when the performance of the CPU changes from point N to point P, the power consumption of the CPU increases the most, that is, when the operating frequency of the CPU changes from point N to point P, the power consumption of the CPU increases the most. In this way, we can select the operating frequency of the CPU corresponding to point N as the specified frequency, so as to minimize the increase in power consumption while increasing the operating frequency.

[0223] Exemplarily, if the performance of the CPU is N and the corresponding operating frequency is 2.13 GHz, then the specified frequency is 2.13 GHz.

[0224] It can be understood that the specified frequency can also be obtained by other means, which is not limited in this application.

[0225] Step S290: When the Render Engine thread finishes calling the CPU to synthesize the target frame, the monitoring sub-thread adjusts the working frequency of the CPU from the specified frequency to the initial working frequency.

[0226] Among them, the specified frequency is greater than the initial working frequency.

[0227] When the duration of the Render Engine thread calling the CPU to synthesize the target frame has been adjusted once, for the next frame after the target frame, it may not be necessary to adjust the working frequency of the CPU. Therefore, after the synthesis thread of the target frame ends, the working frequency of the CPU can be adjusted to the initial working frequency to reduce the power consumption of the CPU. In this way, it is possible to achieve dynamic frequency modulation of the CPU during the image refresh process, and it is possible to increase the frequency when needed to reduce the possibility of frame loss, and it is also possible to take into account the power consumption of the CPU.

[0228] The following combines specific examples to illustrate the dynamic frequency increase method of the CPU provided in this embodiment.

[0229] Figure 14 In it, the synthesis thread of image frame B timed out, resulting in image frame B not being cached in the Buffer before time t2. As a result, during the time period t2 - t3, image frame A is still displayed, and thus a frame loss phenomenon occurs during the time period t2 - t3.

[0230] Figure 17 is the update process diagram of the image frames after applying the dynamic frequency increase method of the CPU with a refresh rate of 60Hz provided in this embodiment. Figure 17 The dashed thread in it represents the update process of the image frames without applying the dynamic frequency increase method of the CPU.

[0231] Combined with Figure 14 and Figure 17 , when adopting the dynamic frequency increase method of the CPU provided in this embodiment, when a frame loss phenomenon occurs during the time period t2 - t3, the time period t2 - t3 can be regarded as the current frame. In the case of a refresh rate of 60Hz, the target historical frame is the previous frame of the current frame, that is, image frame B synthesized during the time period t1 - t2.

[0232] At this time, SurfaceFlinger can obtain an initial threshold based on the duration of the image frame B of the Render Engine thread and determine whether the initial threshold meets the set threshold range. When the initial threshold meets the set threshold range, the initial threshold can be used as the first threshold. SurfaceFlinger submits the first threshold to the monitoring sub-thread, and the monitoring sub-thread monitors the target frame based on the first threshold.

[0233] When frame dropping has occurred in the current frame, the target frame is an image frame that is being synthesized when displayed in the current frame, such as Figure 17 the image frame C in. The synthesis thread of the image frame C includes two stages. The Render Engine thread calls the CPU to perform the first-stage synthesis to form an image frame C'. Among them, the Render Engine thread runs on the CPU. After the image frame C' is synthesized, the CPU submits the image frame C' to the GPU for the second-stage synthesis, thereby forming the image frame C.

[0234] Figure 17 During the first-stage synthesis of the image frame C in, T1 is the first threshold. When the monitoring sub-thread discovers that the duration of the first-stage synthesis reaches T1, it adjusts the working frequency of the CPU. Within T2 time, the CPU works at a specified frequency, thereby adjusting the first-stage synthesis duration from the original T0 to T1 + T2, and T1 + T2 is less than T0. In this way, it is possible to shorten the duration of the Render Engine thread in the target frame by increasing the working frequency of the CPU, thereby effectively reducing the entire synthesis duration of the image frame C, so that the image frame C can be cached to the Buffer before the t4 moment, and then the image frame C is displayed during the t4 - t5 time period.

[0235] It can be seen from Figure 17 that if the working frequency of the CPU in the synthesis thread of the image frame C is not adjusted, the synthesis thread of the image frame C will time out and cannot be cached to the Buffer before the t4 moment, and then the image frame C cannot be displayed during the t4 - t5 time period, resulting in frame dropping occurring again during the t4 - t5 time period. After adopting the method provided in this embodiment, the image frame C can be synthesized, updated, and displayed normally within the specified time, avoiding the risk of frame dropping of the image frame C.

[0236] When the Render Engine thread calls the CPU to complete the first-stage synthesis of the image frame C, the monitoring sub-thread adjusts the working frequency of the CPU to the initial working frequency. In Figure 17In this case, at the end of the time period T2, the operating frequency of the CPU is adjusted to the initial operating frequency to reduce the power consumption of the CPU. In this way, when it is necessary for the CPU to shorten the synthesis thread of the target frame, the operating frequency of the CPU can be increased to reduce the possibility of frame drops. When it is not necessary for the CPU to shorten the synthesis thread of the target frame, the CPU operates at the initial operating frequency, thereby reducing power consumption and saving energy.

[0237] Figure 15 In this case, the synthesis thread of the image frame B times out, resulting in the image frame B not being cached in the Buffer before the time t3, thereby causing the image frame A to still be displayed during the time period t3 - t4, and further causing a frame drop phenomenon during the time period t3 - t4.

[0238] Figure 18 This is the update process diagram of the image frame after applying the dynamic frequency increase method of the application CPU when the refresh rate is 90Hz provided in this embodiment. Figure 18 The dotted line thread in this represents the update process of the image frame without applying the dynamic frequency increase method of the CPU.

[0239] Combined with Figure 15 and Figure 18 , when adopting the dynamic frequency increase method of the CPU provided in this embodiment, when a frame drop phenomenon occurs during the time period t3 - t4, the time period t3 - t4 can be recognized as the current frame. In the case of a refresh rate of 90Hz, the target historical frame is the first two frames before the current frame, that is, the image frame B synthesized during the time period t1 - t2.

[0240] At this time, SurfaceFlinger can obtain the initial threshold based on the duration of the Render Engine thread calling the CPU to synthesize the image frame B, and determine whether the initial threshold meets the set threshold range. When the initial threshold meets the set threshold range, the initial threshold can be used as the first threshold. SurfaceFlinger submits the first threshold to the monitoring sub-thread, and the monitoring sub-thread monitors the target frame according to the first threshold.

[0241] When a frame drop phenomenon has occurred in the current frame, the target frame is the image frame that needs to be synthesized in the next synthesis cycle, such as Figure 18 the image frame C in this. The synthesis thread of the image frame C includes two stages. The Render Engine thread calls the CPU to perform the first-stage synthesis to form the image frame C'. Among them, the Render Engine thread runs on the CPU. After the image frame C' is synthesized, the CPU submits the image frame C' to the GPU for the second-stage synthesis, thereby forming the image frame C.

[0242] Figure 18During the synthesis process of the middle image frame C in the first stage, T1 is the first threshold. When the monitoring sub-thread detects that the duration of the first-stage synthesis reaches T1, it adjusts the working frequency of the CPU. Within the time period T2, the CPU works at a specified frequency, thereby adjusting the duration of the first-stage synthesis from the original T0 to T1 + T2, and T1 + T2 is less than T0. In this way, by increasing the working frequency of the CPU, the duration of the Render Engine thread calling the CPU to synthesize the target frame can be shortened, thereby effectively reducing the overall synthesis duration of the image frame C, enabling the image frame C to be cached in the Buffer before time t4, and then displaying the image frame C in the time period t5 - t6.

[0243] It can be seen from Figure 18 that if the working frequency of the CPU in the synthesis thread of the image frame C is not adjusted, the synthesis thread of the image frame C will time out and cannot be cached in the Buffer before time t5, and then the image frame C cannot be displayed in the time period t5 - t6, resulting in a frame drop phenomenon occurring again in the time period t5 - t6. After adopting the method provided in this embodiment, the image frame C can be synthesized and updated and displayed normally within the specified time, avoiding the risk of frame drop of the image frame C.

[0244] When the Render Engine thread calls the CPU to complete the first-stage synthesis of the image frame C, the monitoring sub-thread adjusts the working frequency of the CPU to the initial working frequency. In Figure 18 this case, at the end of the time period T2, the working frequency of the CPU is adjusted to the initial working frequency to reduce the power consumption of the CPU. In this way, it can be achieved that when the CPU is required to shorten the synthesis thread of the target frame, the working frequency of the CPU is increased to reduce the possibility of frame drop. When the CPU is not required to shorten the synthesis thread of the target frame, the CPU works at the initial working frequency, thereby being able to reduce power consumption and save energy.

[0245] The dynamic frequency increase method of the CPU provided in this embodiment can handle the frame drop phenomenon in a timely manner and adjust the working frequency of the CPU to reduce the duration of the Render Engine thread calling the CPU to synthesize the target frame, thereby being able to reduce the running time of the target frame in the entire synthesis thread, enabling the target frame to be cached in the Buffer at the specified time. In this way, the display driver can normally read the target frame, thereby reducing the possibility of frame drop and improving the smoothness of image refresh and the user experience.

[0246] Figure 19 is a flowchart of another dynamic frequency increase method of the CPU provided in this embodiment.

[0247] As Figure 19As shown in the figure, the dynamic frequency boosting method of the CPU provided in this embodiment includes:

[0248] Step S310: Detect whether the current frame is dropped.

[0249] Wherein, the current frame is the image frame to be displayed on the display screen at the current moment.

[0250] During the normal image frame update process of the electronic device, if the current frame is dropped, SurfaceFlinger in the composition thread can timely grasp the dropped frame situation.

[0251] Step S320: When the current frame is dropped, when the duration for the CPU to compose the target frame is greater than or equal to the first threshold, increase the working frequency of the CPU to the specified frequency.

[0252] Wherein, the target frame is the image frame currently being composed.

[0253] When the current frame is dropped and the image frame fails to be normally displayed, it means that the image frame has not been composed and cached within the specified time.

[0254] It should be noted that the specified frequency is greater than the working frequency of the CPU before adjustment. In this way, when the duration for the CPU to compose the target frame is greater than or equal to the first threshold, the working frequency of the CPU can be increased to reduce the duration for the CPU to compose the target frame, so as to shorten the composition time of the target frame and thus reduce the possibility of the target frame being dropped.

[0255] Exemplarily, the specified frequency of the CPU can be the rated working frequency or the highest working frequency of the CPU, or it can also be other values greater than the working frequency before adjustment, such as 2.13 GHz, etc.

[0256] In some embodiments, in the above steps S310 - S320, when increasing the working frequency of the CPU to the specified frequency, when the composition thread requires the CPU to run, the running frequency of the CPU during the entire process of the composition thread can be increased to 1.5 GHz. This adjustment method helps to reduce the running time of the composition thread, thereby reducing the possibility of subsequent frame dropping.

[0257] In other embodiments, in the above steps S310 - S320, when increasing the working frequency of the CPU to the specified frequency, it is also possible to monitor the duration for the Render Engine thread to call the CPU for composition after detecting frame dropping. When it exceeds 1.2 ms, the working frequency of the CPU is increased to 1.5 GHz. In this way, it can comprehensively cover the composition scenarios of subsequent image frames to reduce the possibility of subsequent frame dropping.

[0258] However, in the above two solutions, although the running time on the CPU can be reduced to a certain extent, the CPU frequency increase experiences more image frames. Even when the frequency increase is not required, the working frequency of the CPU is adjusted, resulting in an increase in the overall power consumption of the CPU, and further affecting the power consumption of the entire synthesis process.

[0259] Figure 20 It is a flowchart of another dynamic CPU frequency increase method provided in this embodiment.

[0260] As Figure 20 shown, in order to reduce the possibility of frame loss while taking power consumption into account, the above step S320 may include:

[0261] Step S410: When a frame loss occurs in the current frame, calculate a first threshold based on the duration of the CPU synthesizing historical frames.

[0262] In an electronic device, when a frame loss phenomenon occurs, the word "framemiss" will be generated in the Surfaceflinger system. When Surfaceflinger detects this information, it can be determined that a frame loss phenomenon has occurred.

[0263] Surfaceflinger can obtain a first threshold based on the duration of the CPU synthesizing historical frames by the Render Engine thread, so as to use the first threshold to monitor the duration of the Render Engine thread in the synthesis thread calling the CPU for synthesis, and shorten the running time of the Render Engine thread calling the CPU to synthesize the target frame.

[0264] Exemplarily, step S410 can be implemented by the above steps S210 - step S251, which will not be elaborated here.

[0265] Step S420: Monitor the duration of the Render Engine thread calling the CPU to synthesize the target frame.

[0266] In order to reduce the probability of frame loss of the target frame after the current frame, the above first threshold can be used to monitor the duration of the Render Engine thread calling the CPU to synthesize the target frame, so as to facilitate timely adjustment of the duration of the Render Engine thread calling the CPU to synthesize the target frame.

[0267] Among them, the monitoring of the Render Engine thread calling the CPU can be implemented by a monitoring sub-thread. After Surfaceflinger obtains the first threshold, Surfaceflinger can send the first threshold to the monitoring sub-thread, so as to use the monitoring sub-thread to monitor the duration of the Render Engine thread according to the first threshold.

[0268] Exemplarily, step S420 can be implemented by the above-mentioned steps S260 - S270, which will not be elaborated here.

[0269] Step S430: When the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, increase the working frequency of the CPU to a specified frequency.

[0270] Among them, the monitoring sub-thread can compare the duration for the Render Engine thread to call the CPU to synthesize the target frame with the first threshold. When the duration for the Render Engine thread to call the CPU to synthesize the target frame exceeds the first threshold, it is necessary to adjust the working frequency of the CPU to reduce the possibility of frame loss of the target frame. When the duration for the Render Engine thread to call the CPU to synthesize the target frame is less than the first threshold, it is not necessary to adjust the working frequency of the CPU, thus avoiding increasing the power consumption of the CPU.

[0271] Exemplarily, step S430 can be implemented by the above-mentioned step S280, which will not be elaborated here.

[0272] Exemplarily, the acquisition of the specified frequency can be implemented by the above-mentioned steps S281 - S283, which will not be elaborated here.

[0273] Alternatively, the specified frequency can also be obtained according to actual experience, which is not limited in this embodiment.

[0274] The dynamic frequency modulation method of the CPU provided in this embodiment can timely adjust the target frame after the frame loss phenomenon occurs. By adjusting the working frequency of the CPU to adjust the running time of the target frame on the CPU, the possibility of frame loss of the target frame can be reduced. In this way, the adjustment of the working frequency of the CPU can be carried out for a certain frame, without requiring the CPU to run at a high working frequency throughout the entire refresh process, which can effectively reduce the power consumption of the CPU.

[0275] The dynamic frequency modulation method of the CPU provided in this embodiment may further include:

[0276] Step S440: When the Render Engine thread finishes calling the CPU to synthesize the target frame, adjust the working frequency of the CPU to the initial working frequency.

[0277] After the Render Engine thread finishes calling the CPU to synthesize the target frame, for the synthesis thread where the target frame is located, the synthesis work of the CPU has been completed. At this time, the working frequency of the CPU can be adjusted to the initial working frequency to avoid the CPU running at a high working frequency for a long time, resulting in high power consumption, thereby saving power.

[0278] Specifically, the monitoring sub-thread can adjust the working frequency of the CPU to the initial working frequency, so that there is no need to re-establish a new thread, simplifying the synthesis system.

[0279] It is worth noting that the above frequency modulation process can be run repeatedly. During the synthesis process of subsequent image frames, if the frame loss phenomenon occurs again, the above method can be repeated to reduce the possibility of the frame loss phenomenon.

[0280] In addition, the dynamic frequency increase method of the CPU provided in this embodiment can achieve adjustment frame by frame, with high flexibility, and while reducing the possibility of frame loss, it can also reduce the increase in power consumption.

[0281] Exemplarily, the dynamic frequency modulation method of the CPU provided in this embodiment can be applied to the scenarios shown in Table 1.

[0282] Table 1

[0283]

[0284]

[0285] It can be understood that only some scenario examples are shown in Table 1 above, and this method can also be applied to other related scenarios, which are not limited in this application.

[0286] Some embodiments of the present application provide an electronic device, which may include: a display screen (such as a touch screen or a non-touch screen), a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute each function or step executed by the electronic device in the above method embodiment. The structure of the electronic device can refer to Figure 8 the structure of the electronic device shown.

[0287] Figure 21 It is a schematic diagram of a chip structure provided in this embodiment.

[0288] The embodiments of the present application also provide a chip system, such as Figure 21As shown, the chip system includes at least one processor 2301 and at least one interface circuit 2302. The processor 2301 and the interface circuit 2302 can be interconnected by a line. For example, the interface circuit 2302 can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit 2302 can be used to send signals to other devices (such as the processor 2301 or the touch screen of an electronic device). Exemplarily, the interface circuit 2302 can read the instructions stored in the memory and send the instructions to the processor 2301. When the instructions are executed by the processor 2301, the electronic device can execute each step in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0289] The embodiments of the present application also provide a computer storage medium, which includes computer instructions. When the computer instructions run on the above-mentioned electronic device, the electronic device is enabled to execute each function or step that the electronic device executes in the above method embodiments.

[0290] The embodiments of the present application also provide a computer program product. When the computer program product runs on a computer, the computer is enabled to execute each function or step that the electronic device executes in the above method embodiments.

[0291] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0292] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application based on several embodiments provided by the present application to obtain other embodiments, and these embodiments do not exceed the protection scope of the present application.

[0293] In the several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0294] The unit described as a separating component may or may not be physically separated. The component shown as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0295] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may be physically present separately for each unit, or two or more units may be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0296] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0297] It should be noted that those skilled in the art will easily think of other implementation schemes of the present application after considering the specification and practicing the application disclosed herein. The present application aims to cover any variations, uses, or adaptive changes of the present application. These variations, uses, or adaptive changes follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and the embodiments are only regarded as exemplary, and the true scope of the present application is pointed out by the claims.

[0298] It should be understood that the present application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A dynamic frequency boosting method for a CPU, characterized in that, Including: Detect whether a frame is lost in the current frame, where the current frame is an image frame to be displayed on the display screen at the current moment; When a frame is lost in the current frame and the duration for the CPU to synthesize the target frame is greater than or equal to a first threshold, increase the working frequency of the CPU to a specified frequency, where the target frame is the image frame currently being synthesized.

2. The dynamic frequency boosting method of the CPU according to claim 1, characterized in that The step of, when a frame is lost in the current frame and the duration for the CPU to synthesize the target frame is greater than or equal to a first threshold, increasing the working frequency of the CPU to a specified frequency includes: When a frame is lost in the current frame, calculate the first threshold based on the duration for the CPU to synthesize historical frames; Monitor the duration for the Render Engine thread to call the CPU to synthesize the target frame; When the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, increase the working frequency of the CPU to the specified frequency.

3. The dynamic frequency boosting method of the CPU according to claim 2, wherein The step of, when a frame is lost in the current frame, calculating the first threshold based on the duration for the CPU to synthesize historical frames includes: When a frame is lost in the current frame, detect the refresh rate of the current image frame; Determine the target historical frame according to the refresh rate of the current image frame; Calculate an initial threshold based on the duration for the Render Engine thread to call the CPU to synthesize the target historical frame; Judge whether the initial threshold is within a set threshold range; If the initial threshold meets the set threshold range, use the initial threshold as the first threshold.

4. The dynamic frequency boosting method of the CPU according to claim 2, characterized in that, The step of monitoring the duration for the Render Engine thread to call the CPU to synthesize the target frame includes: Surfaceflinger sends the first threshold to the monitoring sub-thread; The monitoring sub-thread monitors the duration for the Render Engine thread to call the CPU to synthesize the target frame according to the first threshold.

5. The dynamic frequency boosting method of the CPU according to claim 4, characterized in that, The step of, when the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, increasing the working frequency of the CPU to the specified frequency includes: When the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, the monitoring sub-thread increases the working frequency of the CPU from the initial working frequency to the specified frequency.

6. The dynamic frequency boosting method of the CPU according to claim 4, wherein, After, when the duration for the Render Engine thread to call the CPU to synthesize the target frame is greater than or equal to the first threshold, increasing the working frequency of the CPU to the specified frequency, it further includes: When the Render Engine thread finishes calling the CPU to synthesize the target frame, the monitoring sub-thread adjusts the working frequency of the CPU from the specified frequency to the initial working frequency; Wherein, the specified frequency is greater than the initial working frequency.

7. The dynamic frequency boosting method of the CPU according to claim 6, characterized in that, The method for obtaining the specified frequency includes: Obtain the energy efficiency ratio curve of the CPU; Calculate the slope between two adjacent points in the energy efficiency ratio curve; Based on the maximum value of the slope, determine the specified frequency, where the specified frequency is less than the maximum working frequency of the CPU.

8. The dynamic frequency boosting method of the CPU according to claim 3, wherein Determining a target historical frame according to the refresh rate of the current image frame includes: If the refresh rate of the current image frame is less than 90 Hz, the first historical frame before the current frame is used as the target historical frame; If the refresh rate of the current image frame is greater than or equal to 90 Hz, the second historical frame before the current frame is used as the target historical frame.

9. The dynamic frequency boosting method of the CPU according to claim 3, characterized in that The calculation method of the initial threshold includes: The Render Engine thread multiplies the duration of synthesizing the target historical frame by the CPU by an adjustment coefficient; Wherein, the adjustment coefficient is less than 1, and the adjustment coefficient is at least one.

10. An electronic device, characterized in that, Including: A display screen, a memory, and one or more processors; the display screen and the memory are coupled to the processor; wherein, computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the dynamic frequency boosting method of the CPU according to any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, Including computer instructions, when the computer instructions run on an electronic device, the electronic device is caused to execute the dynamic frequency boosting method of the CPU according to any one of claims 1-9.

Images