Image display method and electronic equipment

Abstract

The invention provides an image display method and electronic equipment, and the method comprises the steps that when the electronic equipment is in a first equipment state, a display screen displays an image at a first refresh rate, and when the electronic equipment is in the first equipment state, the electronic equipment receives a target interaction event, the target interaction event comprises a click event and a first sliding event, and the first sliding event is a sliding event of which the sliding speed is greater than a first preset threshold value; the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold. Visibly, when the electronic equipment is in the first equipment state, the electronic equipment can refresh the content in the display screen at the highest refresh rate, so that the situation that the electronic equipment refreshes the content in the display screen at the highest refresh rate under unnecessary conditions is avoided, and system resources are saved.

Description

Technical Field

[0001] This application relates to the field of terminal devices, and particularly to an image display method and an electronic device. Background Art

[0002] The refresh rate of the display screen of an electronic device refers to the number of times the electronic device refreshes the content displayed on its display screen per unit time, usually measured in Hertz (Hz). Common display screen refresh rates include 60Hz, 90Hz, 120Hz, etc. For example, a 60Hz refresh rate means that the content displayed on the display screen is refreshed 60 times per second. Among them, the content displayed once can be called a frame of image or a frame of picture. By continuously refreshing the content it displays, the display screen can play dynamic content such as videos and animations. Usually, the electronic device can automatically switch the refresh rate according to the requirements of the application program, the user's operations, and the status of the device itself.

[0003] In practical applications, the electronic device can make a decision on the refresh rate based on its interaction state with the user. For example, if the display screen is being touched by the user, that is, the user is interacting with the electronic device, then the electronic device can set the refresh rate of the display screen to the highest refresh rate, such as 120Hz.

[0004] However, if the electronic device refreshes the display screen at the highest refresh rate every time an interaction occurs, it is likely to cause waste of system resources. For example, when the user clicks on the display screen multiple times at a low frequency, there is little animation played on the display screen, and it is not necessary to refresh it at the highest refresh rate, resulting in waste of system resources. Summary of the Invention

[0005] Embodiments of this application provide an image display method and an electronic device to solve the problem of waste of system resources caused by the traditional image display method frequently displaying images at the highest refresh rate.

[0006] In a first aspect, embodiments of this application provide an image display method, which is applied to an electronic device and includes: when the electronic device is in a first device state, the display screen of the electronic device displays an image at a first refresh rate, where the electronic device being in the first device state includes: the electronic device receives a target interaction event, the target interaction event includes: a click event and a first sliding event, and the first sliding event is a sliding event with a sliding speed greater than a first preset threshold; the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold.

[0007] In the image display method provided by the embodiments of the present application, the electronic device refreshes the content on the display screen at the highest refresh rate only when a click event and a first sliding event are received. In this way, the number of times the electronic device refreshes the content on the display screen at the highest refresh rate can be reduced, avoiding unnecessary high-refresh-rate refreshing of the electronic device, and being able to reduce the power consumption of the electronic device.

[0008] In an implementable manner, the electronic device being in the first device state further includes: the display screen is playing a target dynamic effect, where the target dynamic effect is that the change range of the dynamic effect image exceeds a first change value. When the display screen plays the target dynamic effect, the electronic device enters the first device state and refreshes the content on the display screen at the highest refresh rate, which can improve the playing effect of the target dynamic effect and can avoid refreshing the content on the display screen at the highest refresh rate when the display screen plays any dynamic effect, and can reduce the power consumption of the electronic device. Generally speaking, when the change range of the dynamic effect is relatively small, the user's perception is not obvious. At this time, the image can be refreshed at a relatively low refresh rate, while when the change range of the dynamic effect is relatively large, the user may perceive the corresponding image change. In this case, appropriately increasing the refresh rate can improve the display effect of the dynamic effect and enhance the user experience. Therefore, this solution can reduce the power consumption of the electronic device as much as possible without affecting the display of the dynamic effect.

[0009] In an implementable manner, the target dynamic effect being that the change range of the dynamic effect image exceeds a first change value includes: the target dynamic effect is that the change range of the dynamic effect image within a first preset duration exceeds a first change value. The first preset duration and the first change value define the speed of change of the dynamic effect image. In this way, when the display screen plays a dynamic effect image with a relatively fast change speed, the image can be displayed at the highest refresh rate, improving the user experience.

[0010] Specifically, when the change range of the dynamic effect is relatively large within a short period of time, the user may perceive the corresponding image change. In this case, appropriately increasing the refresh rate can improve the display effect of the dynamic effect and enhance the user experience, while when the change range of the dynamic effect is relatively small within a short period of time, the user's perception is not obvious. At this time, the image can be refreshed at a relatively low refresh rate. This solution can reduce the power consumption of the electronic device as much as possible without affecting the display of the dynamic effect.

[0011] In an implementable manner, it further includes: when the electronic device is not in the first device state, the display screen displays an image using a refresh rate matching the image layer. In the embodiments of the present application, using a refresh rate matching the image layer can ensure that the image is smoother and clearer during display, reduce tearing or blurring of the screen, and provide a good visual experience. If the refresh rate is too high, it will cause the system to consume excessive system resources and electrical energy, and no additional visual effects will be brought. Therefore, using a refresh rate matching the image layer to display the image can also avoid unnecessary resource waste and energy consumption.

[0012] In an implementable manner, when the electronic device is not in the first device state, the display screen displays an image using a refresh rate matching the image layer, including: when the electronic device is not in the first device state and the display screen includes M first target layers, the display screen displays the image at a second refresh rate, where the first target layers are active layers, and the matching frame rate of the first target layers is greater than or equal to a first threshold, the active layers include visible layers whose image content has been updated within a first historical duration, and the second refresh rate is determined based on the matching frame rates of the M first target layers; when the electronic device is not in the first device state and the display screen does not include first target layers, the display screen displays the image at a third refresh rate, and the third refresh rate is the default refresh rate of the display screen. When the electronic device is not in the first device state and the display screen includes first target layers, determining the refresh rate at which the display screen displays the image based on the matching frame rates of the first target layers can make the refresh rate match the first target layers and improve the display effect of the first target layers. When the electronic device is not in the first device state and the display screen does not include first target layers, the electronic device displays the image at the default refresh rate, which can provide a good visual experience and avoid excessive power consumption.

[0013] In an implementable manner, the second refresh rate is the maximum refresh rate among refresh rate 1 to refresh rate M, where refresh rate 1 is the refresh rate expected by the first first target layer, refresh rate M is the refresh rate expected by the Mth first target layer, refresh rate 1 is determined based on the matching frame rate of the first first target layer and the optional refresh rates supported by the display screen, and refresh rate M is determined based on the matching frame rate of the Mth first target layer and the optional refresh rates supported by the display screen.

[0014] The embodiments of the present application can calculate the refresh rate expected by each first target layer, and finally take the maximum value among the refresh rates expected by all first target layers. In this way, the refresh effect of each first target layer can reach the best, and there will be no situations such as freezing or tearing.

[0015] In an implementable manner, the default refresh rate is the refresh rate when no adjustment is made to the refresh rate of the image displayed on the display screen. The display screen refreshes the content at the default refresh rate, which can reduce the power consumption of the electronic device and avoid unnecessary resource waste.

[0016] In an implementable manner, the default refresh rate can be preset according to the refresh rates supported by the display screen. Generally speaking, a refresh rate other than the maximum and minimum refresh rates supported by the display screen can be used as the default refresh rate.

[0017] In an implementable manner, the default refresh rate is 60Hz. The 60Hz refresh rate has wide compatibility in the display screen, and the frame rate of many layers is 60 frames per second. Therefore, refreshing the content in the display screen at 60Hz as the default refresh rate can provide a good user experience.

[0018] For example, if the refresh rates supported by the display screen include 30Hz, 60Hz, 90Hz, and 120Hz, then 60Hz can be set as the default refresh rate.

[0019] In a second aspect, an embodiment of the present application provides an image display method, including: before each time the electronic device refreshes the content in the display screen, determining whether the electronic device is in a first device state, where the first device state includes that the electronic device receives a target interaction event or the display screen is playing a target dynamic effect, the target interaction event includes a click event and / or a first sliding event, and the first sliding event is a sliding event with a sliding speed greater than a first preset threshold; when the electronic device is in the first device state, refreshing the content in the display screen at a first refresh rate, where the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold; when the electronic device is not in the first device state, determining whether the display screen includes a first target layer, where the first target layer is an active layer and the matching frame rate of the first target layer is greater than or equal to a first threshold, and the active layer includes a visible layer that has updated the image content within a first historical duration; when the display screen includes at least one first target layer, respectively determining at least one desired refresh rate from at least one optional refresh rate based on the matching frame rate of each first target layer, and refreshing the content in the display screen at the highest refresh rate among the at least one desired refresh rates; when the display screen does not include the first target layer, refreshing the content in the display screen at a third refresh rate, where the third refresh rate is the default refresh rate of the display screen.

[0020] In the image display method provided by the embodiment of the present application, when the electronic device receives a click event and / or a first sliding event, or when the display screen is playing a target dynamic effect, the content on the display screen is refreshed at the highest refresh rate; in other cases, the electronic device can determine the refresh rate used by the display screen by scoring according to the layer state on the display screen from multiple optional refresh rates, or refresh the content on the display screen at the default refresh rate, so as to avoid the electronic device refreshing the content on the display screen at the highest refresh rate under unnecessary circumstances and save system resources.

[0021] In an implementable manner, before determining whether the display screen includes a first target layer, the method further includes: determining whether the display screen includes a second target layer, where the second target layer is an active layer that is playing a dynamic effect; determining whether the display screen includes a first target layer includes: when the display screen does not include the second target layer, determining whether the display screen includes a first target layer. In this way, when the display screen does not include the second target layer, the refresh rate of the image displayed on the display screen can be determined based on the scoring process.

[0022] In an implementable manner, determining at least one expected refresh rate from at least one optional refresh rate based on the matching frame rate of each first target layer respectively includes: for each first target layer, when the matching frame rate is less than or equal to a second threshold, determining the optional refresh rate less than or equal to the second threshold as the candidate refresh rate, where the second threshold is greater than the first threshold; for each first target layer, when the matching frame rate is greater than the second threshold, determining the optional refresh rate greater than the second threshold as the candidate refresh rate; scoring each candidate refresh rate based on the matching frame rate; and determining the candidate refresh rate with the highest score as the expected refresh rate of the first target layer. In this way, an expected refresh rate close to the matching frame rate of the layer can be selected, and it is also possible to avoid scoring all optional refresh rates based on the matching frame rate, which can reduce the calculation amount and quickly determine the refresh rate of the image displayed on the display screen.

[0023] In an implementable manner, scoring each candidate refresh rate based on the matching frame rate includes: when the candidate refresh rate is equal to the matching frame rate, determining the candidate refresh rate as a first score; when the candidate refresh rate is not equal to the matching frame rate, determining the candidate refresh rate as a second score, where the second score is less than the first score; or, when the candidate refresh rate is an integer multiple of the matching frame rate, determining the candidate refresh rate as a first score; when the candidate refresh rate is not an integer multiple of the matching frame rate, determining the candidate refresh rate as a second score. In this way, the expected refresh rate selected by scoring can be made as close as possible to the matching frame rate of the layer, improving the display effect of the layer.

[0024] In an implementable manner, it further includes: before each refresh of the content on the display screen of the electronic device, obtaining a cache result, where the cache result includes the device status information of the electronic device, the layer status information of the active layer, and the historical refresh rate information adopted by the display screen before the last refresh of the content on the display screen. Among them, the screen refresh rate was the first historical refresh rate when the electronic device last refreshed the content on the display screen; determining whether the number of current active layers is the same as the number of active layers before the last refresh of the content on the display screen; in the case where the number of active layers is different, determining the number of optional refresh rates; in the case where the number of active layers is the same, determining whether the layer status information of the current active layer is the same as the layer status information of the active layer before the last refresh of the content on the display screen, and determining whether the device status information of the current electronic device is the same as the device status information of the electronic device before the last refresh of the content on the display screen; in the case where the layer status information of the active layer is the same and the device status information of the electronic device is the same, refreshing the content on the display screen at the first historical refresh rate. In this way, determining the refresh rate based on the cache result can reduce the computational load.

[0025] In an implementable manner, it further includes: in the case where the layer status information of the active layer is different or the device status information of the electronic device is different, determining the number of optional refresh rates; in the case where there is only one optional refresh rate, refreshing the content on the display screen at the optional refresh rate. In this way, the computational load is reduced.

[0026] In an implementable manner, it further includes: in the case where there are multiple optional refresh rates, determining whether each layer on the display screen is in a static state and the electronic device has not received a touch event; in the case where each layer on the display screen is in a static state and the electronic device has not received a touch event, refreshing the content on the display screen at the lowest refresh rate among the multiple optional refresh rates; in the case where at least one layer on the display screen is in a non-static state or the display screen has received a touch event, determining whether the active layer only includes a third target layer; where the third target layer includes a wallpaper layer, and / or, a layer with a matching frame rate less than a first threshold, and a status bar layer; in the case where the active layer only includes the third target layer, refreshing the content on the display screen at the lowest refresh rate among the multiple optional refresh rates. In the case where the active layer includes the third target layer and also includes other layers, determining whether the electronic device is in a first device state. In this way, the refresh rate of the display image on the display screen can be determined based on the device status information and the status information of the active layers on the display screen.

[0027] In an implementable manner, it further includes: when the electronic device receives a target interaction event or the display screen is playing a target dynamic effect, it enters the first device state and starts the first timer, and the first timer is set with a first timing duration; when the timing of the first timer exceeds the first timing duration, and / or when the electronic device receives a second sliding event, and / or when all layers in the display screen are in a static state, and / or when the target dynamic effect ends, the electronic device exits the first device state, and the second sliding event is a sliding event with a sliding speed less than or equal to the first preset threshold. In this way, it can be determined whether the electronic device is in the first device state.

[0028] In an implementable manner, before obtaining the cache result, it further includes: before the electronic device refreshes the content in the display screen each time, for each active layer, it determines whether the active layer contains matching frame rate information, and the matching frame rate information is used to indicate the matching frame rate; when the active layer does not contain the matching frame rate information, the matching frame rate is determined based on the number of updates of the active layer within the second historical duration and the time interval of the updates. Determining the matching frame rate based on the number of updates of the active layer within the second historical duration and the time interval of the updates includes: when the number of updates is greater than or equal to the number threshold or the time interval of each update is equal, the matching frame rate is the ratio of the number of updates to the second historical duration; when the number of updates is less than the number threshold and the time interval of each update is not equal, the matching frame rate is the first preset value. In this way, the matching frame rate of the first target layer can be calculated based on this.

[0029] In an implementable manner, it further includes: saving the device state information, layer state information before the electronic device refreshes the content in the display screen each time, and the refresh rate adopted by the display screen when the electronic device refreshes the content in the display screen each time. In this way, the cache result corresponding to this image display can be formed for use when determining the refresh rate of the image display next time.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory, and the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image display method as described in the first aspect and any implementation manner or the second aspect and any implementation manner above.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and instructions are stored in the computer-readable storage medium, and when the instructions run on the electronic device, the electronic device executes the image display method as described in the first aspect and any implementation manner or the second aspect and any implementation manner above.

[0032] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device is caused to execute the image display method in the first aspect and any implementation manner thereof or the second aspect and any implementation manner thereof as described above.

[0033] It can be understood that the electronic devices, computer-readable storage media, and computer program products provided in the above aspects are all applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the composition of the electronic device layers;

[0035] Figure 2 It is the first schematic diagram of the refresh rate decision model of the electronic device;

[0036] Figure 3 It is a schematic logical diagram of the electronic device switching system states shown in the embodiment of the present application;

[0037] Figure 4 It is the second schematic diagram of the refresh rate decision model of the electronic device;

[0038] Figure 5 It is the third schematic diagram of the refresh rate decision model of the electronic device;

[0039] Figure 6 It is a schematic diagram of the large-weight layer and the small-weight layer;

[0040] Figure 7 It is a schematic diagram of multiple layers participating in the decision-making;

[0041] Figure 8 It is a schematic hardware structure diagram of the electronic device 100 provided in the embodiment of the present application;

[0042] Figure 9 It is a schematic software structure block diagram of the electronic device 100 provided in the embodiment of the present application;

[0043] Figure 10 It is the first flow schematic diagram of the image display method provided in the embodiment of the present application;

[0044] Figure 11 It is a schematic diagram of the target interaction event provided in the embodiment of the present application;

[0045] Figure 12 It is the second flow schematic diagram of the image display method provided in the embodiment of the present application;

[0046] Figure 13The first schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application;

[0047] Figure 14 The second schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application;

[0048] Figure 15 The first application schematic diagram of the image display method provided by the embodiments of the present application;

[0049] Figure 16 The schematic diagram of the display screen provided by the embodiments of the present application that does not include the first target layer;

[0050] Figure 17 The third schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application;

[0051] Figure 18 The second application schematic diagram of the image display method provided by the embodiments of the present application;

[0052] Figure 19 The third process schematic diagram of the image display method provided by the embodiments of the present application;

[0053] Figure 20 The fourth schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application;

[0054] Figure 21 The third application schematic diagram of the image display method provided by the embodiments of the present application;

[0055] Figure 22 The fourth application schematic diagram of the image display method provided by the embodiments of the present application;

[0056] Figure 23 The fifth schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application;

[0057] Figure 24 The process schematic diagram for comparing with the cache result provided by the embodiments of the present application;

[0058] Figure 25 The flow chart for determining the layer type provided by the embodiments of the present application;

[0059] Figure 26 For Figure 25 An example diagram of the shown process;

[0060] Figure 27 The flow chart for scoring the optional refresh rate provided by the embodiments of the present application;

[0061] Figure 28 For Figure 27 An example diagram of the process shown;

[0062] Figure 29 A schematic diagram of an electronic device switching its device state provided by an embodiment of the present application;

[0063] Figure 30 It is a schematic structural diagram of an image display device provided by an embodiment of the present application. Detailed implementation manners

[0064] Next, the technical solutions of the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application.

[0065] In the description of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. Herein, "and / or" is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. The words such as "first" and "second" do not limit the quantity and execution order, and the words such as "first" and "second" do not necessarily limit to be different.

[0066] It should be noted that in the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present related concepts in a specific manner.

[0067] The refresh rate of the display screen refers to the number of times an electronic device updates the image displayed on the display screen per unit time, usually measured in Hertz (Hz). Common refresh rates of display screens include 60Hz, 90Hz, 120Hz, etc. Exemplarily, a 60Hz refresh rate means that the electronic device can refresh the display content 60 times per second, and the content displayed once can be called a frame of image or a frame of picture.

[0068] As the display screen is continuously refreshed, it can continuously display new image data to show continuous image changes. Generally speaking, the higher the refresh rate of the display screen, the smoother the image display, and the fewer the stuttering and flickering phenomena. In addition, a high refresh rate can also reduce the generation of image motion blur. Image motion blur refers to the blurred or trailing effect between consecutive frames due to insufficient refresh rate in the fast-moving elements displayed on the display screen. Among them, the fast-moving elements are, for example, the fast-moving game characters in the game screen.

[0069] Therefore, increasing the refresh rate of the display screen can improve the visual experience of the display screen and enhance the user experience.

[0070] However, in electronic devices such as mobile phones, the display screen does not always work at the highest refresh rate. This is because: the higher the refresh rate of the display screen, the higher the system resources consumed by the electronic device for rendering and synthesizing images. Therefore, the electronic device can provide a refresh rate automatic adjustment mechanism, so that the electronic device can automatically adjust the refresh rate of the display screen to adjust the resource consumption degree of the process of displaying images, and at the same time enhance the user experience.

[0071] The following is an exemplary introduction to the general method of automatically switching the refresh rate of the electronic device.

[0072] The mechanism of automatically switching the refresh rate of the electronic device is usually referred to as Adaptive Refresh Rate or Intelligent Refresh Rate. Specifically, it can dynamically adjust the refresh rate of the display screen according to the content currently displayed on the display screen. For example, when a static image is displayed on the display screen of the electronic device or when it is in the SMS editing interface, the electronic device can reduce the refresh rate to save battery power. When a video or game interface is being played on the display screen of the electronic device, the electronic device can increase the refresh rate to obtain a smoother picture effect and response speed.

[0073] Generally speaking, the mobile phone display screen has multiple refresh rate options (such as 60Hz, 90Hz, 120Hz), and the operating system switches according to needs. Applications, etc. will also control the refresh rate to ensure that the image content they output can adapt to different refresh rates. The benefit of automatically switching the refresh rate is that it can reduce power consumption while ensuring a good display effect and extend the battery life.

[0074] In mobile display technology, all application programs and system interfaces exist in the form of "layers". An electronic device can stack these "layers" according to attributes such as priority, transparency, and position, and finally form the content displayed on the display screen. Each of these "layers" is a layer. In mobile display technology, each layer can contain different images, texts, icons, or other visible elements. Each layer on the mobile display screen can be processed and rendered independently and then superimposed to form the final display image. By placing different elements on different layers, more efficient, flexible, and smooth image presentation can be achieved.

[0075] The following specifically introduces the types and styles of layers, as well as the composition and display processes of layers.

[0076] Figure 1 It is a schematic diagram of the composition of layers for an electronic device.

[0077] Such as Figure 1 shown, common mobile display layers include the following:

[0078] Application Layer: It contains the main display content of the application program, such as the application interface, images, videos, etc. Figure 1 The layer of the recorder application is exemplarily shown in

[0079] System Layer: It contains system-level elements such as the user interface of the operating system, status bar, and notification bar.

[0080] Launcher: Usually located at the bottom layer of all other layers. It contains application icons, folders, etc., and is responsible for handling touch operations or gestures on these elements by the user.

[0081] Wallpaper Layer: Responsible for displaying the desktop wallpaper.

[0082] From the generation of a layer to its display on the display screen, it generally includes the following stages: creation and drawing stage, rendering stage, layer composition stage, frame buffer storage stage, display controller processing stage, and display screen refresh stage.

[0083] Taking the application layer as an example, the steps from the generation to the display of the layer are introduced. The processes for an application program to display graphics on the mobile display screen include the following:

[0084] S1, Layer Creation and Drawing: The application program first creates layer data such as graphic objects, textures, or bitmaps through a drawing API (such as OpenGL ES), and then submits them to the SurfaceFlinger.

[0085] S2, Graphics Processing Unit (GPU) Rendering: After the application submits the layer data to SurfaceFlinger, SurfaceFlinger passes them to the GPU for actual graphics rendering. The GPU can utilize its graphics processing capabilities to process and render these graphic objects, such as vertex transformation, texture mapping, lighting calculation, etc., to generate the final image.

[0086] S3, Image Composition: After the GPU finishes rendering, the generated image is returned to SurfaceFlinger in the form of an image buffer. Continue to refer to Figure 1 , SurfaceFlinger is responsible for compositing multiple layers. The composition process involves operations such as blending different layers, transparency processing, and image transformation, etc., to generate the final image to be displayed on the display screen.

[0087] S4, Frame Buffer Storage Stage: After the composition is completed, SurfaceFlinger can store the final image data in the frame buffer.

[0088] S5, Display Controller Processing: The display controller is responsible for reading the image data from the frame buffer and sending the data to the display screen.

[0089] S6, Display Screen Refresh: The display screen can refresh each pixel segment in a certain order based on the received image data, thereby displaying the image.

[0090] The above process is continuously carried out. When the application needs to refresh a certain layer, it will send the image to the GPU again to keep the content on the display screen updated in real time.

[0091] Generally speaking, the electronic device can determine the refresh rate (Best Refresh Rate) when refreshing the display screen each time based on the refresh rate decision model.

[0092] Figure 2 It is a schematic diagram of the architecture of a refresh rate decision model shown in the embodiments of the present application.

[0093] As Figure 2 shown, in this refresh rate decision model, the refresh rate is mainly determined by three factors: the available refresh rate of the electronic device, the layer state of the layers in the display screen, and the system state.

[0094] Among them, the available refresh rates include one or more display refresh rates supported by the electronic device, which are generally determined by factors such as the operating system of the electronic device, the performance of the display screen, and the performance of the image processor. For example, the electronic device can support four available refresh rates: Rate1, Rate2, Rate3, and Rate4. For example, Rate1, Rate2, Rate3, and Rate4 can be 30Hz, 60Hz, 90Hz, and 120Hz respectively. In practical applications, the operating system usually provides some interfaces or APIs to allow applications to express their expectations or requirements for the refresh rate to the system.

[0095] Continue to refer to Figure 2 , the layer state includes layer type, layer voting type, layer weight, and layer desired frame rate (desiredFps).

[0096] Taking the Android operating system as an example, layers can generally be divided into the following types:

[0097] StatusBar layer, Wallpaper layer: Among them, both the StatusBar layer and the Wallpaper layer belong to system layers.

[0098] Minimum refresh rate layer (MIN): Such as Figure 2 layer-0 shown in, including layers with a relatively low drawing frequency of the application, such as less than 10Hz. The drawing frequency refers to the frequency at which the application sends layer data to SurfaceFlinger.

[0099] Maximum refresh rate layer (MAX): Such as Figure 2 layer-2 shown in, including the layer corresponding to the startup of the application or the layer involving a large range of dynamic effects. The desired frame rate of the maximum refresh layer is equal to the highest refresh rate among the available refresh rates.

[0100] Specified frame rate layer (Explicit): Such as Figure 2 layer-1 shown in, including layers for which the application has set the desired frame rate.

[0101] Statistical layer (Heuristic): Such as Figure 2 layer-3 shown in, including layers for which the drawing frequency can be statistically calculated. The Android system can set the desired frame rate for the layer based on the drawing frequency.

[0102] Layer that cannot be statistically calculated: Such as Figure 2 layer-4 shown in, including layers for which the drawing frequency cannot be statistically calculated.

[0103] The layer weight can refer to the proportion of the layer size to the display screen size.

[0104] Continue to refer to Figure 2 The system state refers to the state of the electronic device determined based on user operations, including the touch state and the non-touch state, as well as the idle state and the non-idle state.

[0105] Figure 3 This is a logical schematic diagram showing the switching of the system state of the electronic device illustrated in the embodiments of the present application.

[0106] Among them, Figure 3 Figure (a) shows the switching logic between the touch state and the non-touch state of the electronic device. For example, when the electronic device is in the non-touch state, the electronic device can enter the touch state when receiving an input event. The input event is, for example, that the user touches the mobile phone display screen, the mobile phone display screen switches from portrait display to landscape display, or switches from landscape display to portrait display, etc. After the electronic device enters the touch state, a touch timer can be started. If no new input event arrives when the timer expires, the electronic device will enter the non-touch state.

[0107] Figure 3 Figure (b) shows the switching logic between the idle state and the non-idle state of the electronic device. For example, when the electronic device is in the idle state, the electronic device can enter the non-idle state when receiving an input event. The input event is, for example, the movement of elements in the display screen. When the electronic device is in the non-idle state, the electronic device can start an idle timer when the display screen is idle. The display screen being idle means that there are no moving elements in the display screen. If there is no element movement or the user does not touch the display screen when the timer expires, the system state will switch from the non-idle state to the idle state.

[0108] Based on Figure 2 the refresh rate decision model shown, the electronic device can first determine whether the foreground application has set the highest refresh rate or the frame rate, that is, whether there are layers of the MAX type or the Explicit type.

[0109] If there are layers of the MAX type or the Explicit type, the electronic device scores one or more available refresh rates based on the layer state of the layers in the display screen, and switches the refresh rate of the display screen to the refresh rate with the highest score. The specific scoring process will be described in detail below and will not be elaborated here.

[0110] Figure 4 This is the second schematic diagram of the refresh rate decision model of the electronic device.

[0111] AsFigure 4 As shown, if there is no layer of the MAX type or the Explicit type, the electronic device can make a decision on the refresh rate based on the system state. Specifically, if the system state is the touch state, then the refresh rate is determined as the highest refresh rate among the available refresh rates, such as 120 Hz. If the system state is the stationary state, then the refresh rate is determined as the lowest refresh rate among the available refresh rates, such as 30 Hz.

[0112] Since the electronic device usually enters the touch state when receiving an input event, this decision-making method is closely related to the interaction behavior between the user and the electronic device. That is to say, as long as the electronic device receives an input event, it will switch the refresh rate of the display screen to the highest refresh rate. However, in some scenarios, the display screen does not need to work at the highest refresh rate. For example, when the user clicks on the display screen multiple times at a relatively slow speed, the dynamic effect generated by this interaction method is very small and does not require refreshing at the highest refresh rate. Therefore, this decision-making method will cause waste of system resources.

[0113] Figure 5 This is the third schematic diagram of the refresh rate decision model for the electronic device.

[0114] As Figure 5 shown, when the electronic device is in a non-interactive state and a non-stationary state at the same time, such as when the electronic device is playing a video, the electronic device can score and vote on one or more available refresh rates based on the layer state, and switch the refresh rate of the display screen based on the scoring and voting results.

[0115] Among them, in the scoring stage, each layer participating in the scoring in the display screen of the electronic device scores each available refresh rate. After the electronic device finishes scoring based on each layer participating in the scoring, the scores of each available refresh rate are summed to obtain the final score of each available refresh rate. The available refresh rate with the highest score is used as the desired refresh rate of each layer participating in the scoring. The specific scoring process of the layer will be described in detail below. It should be noted that the layers with the desired frame rate (layer-1, layer-2, layer-3) participate in the scoring process, and the layer (layer-4), the lowest refresh rate layer (layer-0), the status bar layer, and the wallpaper layer do not participate in the scoring process.

[0116] The voting stage refers to the electronic device putting each layer into the voting process with the desired refresh rate of the layer. It can be understood that after the scoring process is completed, the electronic device can determine the desired refresh rate of each layer participating in the scoring. For the lowest refresh rate layer and the wallpaper layer, the desired refresh rate set by the Android system for them is the lowest refresh rate among the available refresh rates. For the layer that cannot be counted, the desired refresh rate set by the Android system for it is the highest refresh rate among the available refresh rates.

[0117] In the voting stage, the electronic device can score the available refresh rates based on the desired refresh rates of each layer in the display screen. For example, if the display screen includes a layer with the lowest refresh rate, and the desired refresh rate of the lowest-refresh-rate layer is the lowest available refresh rate, then the lowest available refresh rate can be scored 1 point. Another example is that if the display screen includes a layer for which the refresh rate cannot be counted, and the desired refresh rate of this layer is the highest available refresh rate, then the highest available refresh rate can be scored 1 point. After that, the electronic device can refresh the display screen with the available refresh rate that has the highest voting score.

[0118] In the actual decision-making process, the layers participating in the voting depend on which layers are actually included in the display screen of the electronic device. For example, if the electronic device does not include the layers (layer-1, layer-2, layer-3) for which scores are calculated and does not include the layer (layer-4) for which the frame rate cannot be counted, then ultimately only the lowest-refresh-rate layer (layer-0) and the wallpaper layer participate in the voting.

[0119] The scoring stage includes: The electronic device scores the available refresh rates based on the desired frame rates of each layer. The score range is [0, 1], and the score size is determined by the difference between the desired frame rate and the available refresh rate. The closer the available refresh rate is to the desired frame rate, the higher the score. When the available refresh rate is equal to the desired frame rate, the score of the available refresh rate is 1. After that, the scoring result is multiplied by the layer weight of the layer to obtain one of the scores of the available refresh rate. After the scoring process of all the layers participating in the scoring by the electronic device is completed, the scores of each available refresh rate are summed to obtain the final score of each available refresh rate in the scoring stage.

[0120] For the optional refresh rate of 120Hz:

[0121] ① The electronic device scores 120Hz based on the desired frame rate of layer-1, and the score is x1, where x1 ranges from [0, 1]. The final value of scoring 120Hz based on layer-1 is x1 * weight.

[0122] ② The electronic device scores 120Hz based on the desired frame rate of layer-2, and the score is x2, where x2 ranges from [0, 1]. The final value of scoring 120Hz based on layer-2 is x2 * weight.

[0123] ③ The electronic device scores 120Hz based on the desired frame rate of layer-3, and the score is x3, where x3 ranges from [0, 1]. The final value of scoring 120Hz based on layer-3 is x4 * weight.

[0124] The final score of 120Hz in the scoring stage = x1 * weight + x2 * weight + x4 * weight.

[0125] The scoring process for other available refresh rates is the same as the above process and will not be elaborated here.

[0126] When the electronic device scores based on the expected frame rate of the layer, the closer the available refresh rate is to the expected frame rate, the higher the score. Also, since the final score of the available refresh rate in the scoring stage is closely related to the weight of the layer. The available refresh rate close to the expected frame rate of the layer with a large weight gets a relatively high score, and the available refresh rate close to the expected frame rate of the layer with a small weight gets a relatively low score. Then, letting the layer weight participate in the refresh rate decision is likely to cause problems such as stuttering.

[0127] Figure 6 It is a schematic diagram of the large-weight layer and the small-weight layer.

[0128] As Figure 6 shown in (a) below, the expected frame rate of the layer with a larger weight is 30Fps when playing a video. Since its weight is large, the score given by this layer to 30Hz is large, which will result in a relatively high final score for 30Hz. The expected frame rate of the layer with a smaller weight is 60Fps when expecting to play a video. Since its weight is small, the score given by this layer to 60Hz is small, which will result in a relatively low final score for 60Hz. The above process causes the two layers to participate in the voting process with 30Hz as the requested refresh rate, and then the decision result is 30Hz. However, as Figure 6 shown in (b) below, this decision result will cause both the large layer and the small layer to be refreshed at 30Hz, ultimately resulting in stuttering in the video playback of the small layer.

[0129] Figure 7 It is a schematic diagram of multiple layers participating in the decision.

[0130] It should also be noted that when there are multiple layers participating in the scoring, for the scoring process of the same available refresh rate, each layer participating in the scoring participates. That is to say, each layer participates in the calculation process of the requested refresh rate of other layers, which is likely to lead to inaccurate requested refresh rates of the layers. Taking the weights of each layer being equal and equal to 0.5 as an example, there are the following scoring scenarios:

[0131] As Figure 7 shown in (a) below, the expected frame rate of layer 1 is 120Fps, 120Hz gets 1 * 0.5 = 0.5 points, and other available refresh rates get 0 points;

[0132] The expected frame rate of layer 2 is 60Fps, 60Hz gets 1 * 0.5 = 0.5 points, and other available refresh rates get 0 points;

[0133] The desired frame rate of Layer 3 is 60 Fps. For 60 Hz, 1 * 0.5 = 0.5 points are obtained, and 0 points are obtained for other available refresh rates.

[0134] In the total score obtained in this scoring stage, 60 Hz (1 point) > 120 Hz (0.5 points) > other available refresh rates. In this way, the desired refresh rate of each layer participating in the scoring is 60 Hz. As Figure 7 shown in (b) below, after each layer votes with its desired refresh rate, the final decision result will be 60 Hz, resulting in a sub-optimal animation experience for Layer 1. It can be seen that each layer participates in the calculation process of the desired refresh rate of other layers, ultimately leading to a poor animation experience.

[0135] In addition, an electronic device usually sets the desired refresh rate of layers without a specified frame rate and for which the image sending frequency cannot be counted to the highest refresh rate. Then, if these layers also participate in the voting, it may cause the highest refresh rate to obtain the most votes, causing the electronic device to switch the refresh rate of the display screen to the highest refresh rate. However, layers without a specified frame rate and for which the image sending frequency cannot be counted are usually some layers that suddenly appear on the display screen and contain irregular animations. When these layers appear, therefore, if the refresh rate of the display screen is switched to the highest refresh rate, it will result in a waste of system resources.

[0136] To solve the above problems, an embodiment of the present application provides an image display method, which can be applied to an electronic device. This solution can avoid resource waste during the refresh rate display process of the display screen, and at the same time can avoid poor layer refresh effects, and can improve the user experience.

[0137] The image display method provided by the embodiment of the present application can be applied to an electronic device with a display function. Among them, the electronic device includes but is not limited to mobile phones, tablet computers, personal computers, workstation devices, large-screen devices (such as smart screens, smart TVs, etc.), wearable devices (such as smart bracelets, smart watches), handheld game consoles, home game consoles, virtual reality devices, augmented reality devices, mixed reality devices, etc., in-vehicle intelligent terminals, etc.

[0138] Figure 8 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present application.

[0139] As Figure 8As shown, the electronic device 100 may include a processor 110, a memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, a camera 192, a display screen 193, and a subscriber identification module (SIM) card interface 194, etc.

[0140] It can be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0141] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a display controller 195, a graphics processing unit (GPU) 196, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0142] The graphics processing unit 196 may be used for image rendering, video decoding and encoding, etc. The display controller 195 may be used to receive image data from the graphics processing unit 196 and convert it into a signal capable of driving the display screen 193 to display.

[0143] The memory 120 can be used to store computer-executable program codes, and the executable program codes include instructions. The memory 120 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the memory 120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the memory 120, and / or the instructions stored in the memory provided in the processor.

[0144] The USB interface 130 is an interface that conforms to the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. This interface can also be used to connect other electronic devices, such as an AR device, etc.

[0145] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0146] The charging management module 140 is used to receive a charging input from a charger. Among them, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input of the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0147] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the memory 120, the display screen 193, the camera 192, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery charge cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0148] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0149] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0150] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.

[0151] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 193. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0152] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be transmitted from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0153] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, such that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0154] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0155] Figure 9 It is a software structure block diagram of the electronic device in the embodiments of the present application.

[0156] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, Android runtime and system libraries, and the kernel layer.

[0157] The application layer may include a series of application packages.

[0158] As Figure 9 shown, the application packages may include applications such as battery management, camera, gallery, calendar, call, map, navigation, music, video, short message, etc.

[0159] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0160] As Figure 5 shown, the application framework layer may include a window manager, an input manager (InputManager), a sensor manager (SensorManager), a telephone manager, a resource manager, a notification manager, etc.

[0161] The input manager can be used to listen for user input events, such as click events, swipe events, etc. performed by the user's finger on the display screen 193 of the electronic device 100. By listening for input events, the electronic device 100 can determine whether the electronic device is being used.

[0162] The sensor manager is used to listen for data returned by various sensors in the electronic device, such as motion sensor data, proximity light sensor data, temperature sensor data, etc. Using the data returned by each sensor, the electronic device can determine whether it is jittery, or whether the display screen 193 is blocked, etc.

[0163] Android Runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

[0164] The core library contains two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android.

[0165] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0166] The system library may include multiple functional modules. For example: surface manager, Media Libraries, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.

[0167] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0168] The media library supports the playback and recording of a variety of common audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0169] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0170] The 2D graphics engine is the drawing engine for 2D drawing.

[0171] Furthermore, the system library also includes a SurfaceFlinger, which is used to manage and synthesize the graphical interfaces of applications. For example: an application can provide layer data to the SurfaceFlinger. After receiving the layer data from the application, the SurfaceFlinger can pass the layer data to the GPU 196 so that the GPU 196 can perform image rendering. After that, the SurfaceFlinger can synthesize the rendered images to obtain multiple layers. When the layer synthesis is completed, the SurfaceFlinger can send the image to the frame buffer.

[0172] The kernel layer is the layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, a sensor driver, and a graphics driver.

[0173] Among them, the display driver can be responsible for processing the image data from the graphics processor 196 in the frame buffer and converting it into a signal recognizable by the display screen 193 to ensure that the correct image can be sent to the display screen 193 and presented in an appropriate manner. The graphics driver can be used to manage and control the graphics processing unit 196 in the computer system and the related graphics functions and display devices.

[0174] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0175] Figure 10 This is the first process schematic diagram of the image display method provided by the embodiments of the present application.

[0176] As shown Figure 10 in the figure, the image display method provided by the embodiment of the present application may include S10: when the electronic device is in the first device state, the display screen of the electronic device displays an image at the first refresh rate, where the electronic device being in the first device state includes: the electronic device receives a target interaction event, and the target interaction event includes: a click event and a first sliding event, and the first sliding event is a sliding event with a sliding speed greater than a first preset threshold; the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold.

[0177] The embodiment of the present application may determine whether the display screen refreshes the image at the first refresh rate based on the device state of the electronic device. The device state may specifically be the first device state, also known as the Boost state. The condition for the electronic device to enter the first device state may be that a specific event occurs, and this specific event may specifically be triggered by the touch sensor of the electronic device capturing various operations of the user on the display screen.

[0178] Specifically, the click event is triggered by a click operation, but not every touch action of the user on the display screen can trigger the click event. An action with an interval less than a certain time threshold between the user's pressing action and lifting action can trigger the click event. For example, as shown Figure 11 in (a) below, the user's finger quickly presses and lifts on the game interface, and at this time the click event occurs. The time threshold is, for example, 300 ms, and the embodiment of the present application does not make specific limitations on this.

[0179] The first sliding event may be triggered by the user's operation of quickly sliding a finger on the display screen, such as when the user quickly browses a long page or quickly scrolls a list. The first sliding event is a sliding event with a sliding speed greater than the first preset threshold. In practical applications, the sliding speed may be determined based on the number of sliding pixels and time of the sliding operation. Specifically, the electronic device may record the positions touched by the finger at a fixed time interval, and calculate the position of the finger on the display screen after m time intervals based on these position information, obtain the pixel difference between the first position and the mth position, and then calculate the sliding speed based on the pixel difference and the total time corresponding to the two positions. The time interval is, for example, 1 ms, and m may be equal to 5, for example. The embodiment of the present application does not make specific limitations on this. Then, based on the sliding speed, it can be calculated whether the sliding event is the first sliding event. For example, as shown Figure 11 in (b) below, the user quickly flips the page, and at this time the first sliding event may occur.

[0180] The first preset threshold may be, for example, 1000 pixels / second or 500 pixels / second, and may be specifically determined according to the actual situation. The embodiment of the present application does not make specific limitations on this.

[0181] The first refresh rate can be determined based on all the refresh rates supported by the display screen. For example, if the display screen supports any one of 30Hz, 60Hz, 120Hz, and 144Hz, then the electronic device can determine the first refresh rate as the highest refresh rate supported by the display screen at this time, which is 144Hz.

[0182] In the embodiments of the present application, the first refresh rate can be determined based on the selectable refresh rates supported by the display screen, and the first refresh rate is one of the selectable refresh rates. The selectable refresh rates supported by the display screen can be jointly determined by the operating state of the electronic device and the hardware capabilities of the display screen. For example, the hardware capabilities of the display screen support a refresh rate of 144Hz or higher, but the electronic device locks the refresh rate range based on factors such as its operating state, so that the display screen refreshes at a maximum refresh rate of 120Hz. At this time, the selectable refresh rates when the display screen refreshes are less than or equal to 120Hz, then the first refresh rate can be 120Hz. Figure 11 In (a) and (b), it is exemplarily shown that the first refresh rate is equal to 120Hz.

[0183] In the embodiments of the present application, the refresh rate threshold can be 60Hz, 90Hz, and the embodiments of the present application do not make specific limitations on this.

[0184] It can be understood that when the user uses the electronic device, many events can be triggered as the interaction progresses. In addition to the click event and the first swipe event, it also includes, for example, the long-press event, the screen orientation change event, etc. Since not all events can trigger the first device state, when other events other than the click event and the first swipe event occur, the electronic device will not enter the first device state. At this time, it is possible to prevent the display screen from displaying images at the first refresh rate. Therefore, based on the method provided by the embodiments of the present application, the number of times the electronic device refreshes the content on the display screen at the highest refresh rate can be reduced, and the electronic device can be prevented from refreshing at the highest refresh rate under unnecessary circumstances, which can reduce the power consumption of the electronic device.

[0185] In the embodiment of the present application, the electronic device being in the first device state may further include: the display screen is playing a target dynamic effect, where the target dynamic effect may be that the change range of the dynamic effect image exceeds a first change value. The change range may refer to the degree or amplitude of the change of the attributes or features of the image, such as the change range of the size of the elements in the image, the change range of the position, the change range of the color, or the change range of the transparency, etc. For example, if a button in the image is enlarged, then the change range of the size of the button is the difference between the enlarged size and the initial size, and at this time the first change value is the size threshold. Another example is that the button in the image is located in the lower left corner of the display screen in the initial state, and after a certain period of time, the button is located in the middle of the display screen. Then the change range of the position is the distance covered by the moving path of the button from the lower left corner to the middle position, and at this time the first change value is the distance threshold. The change range of the position can be measured in different dimensions, such as horizontal lateral movement and vertical longitudinal movement.

[0186] In the embodiment of the present application, the target dynamic effect may specifically be a splash screen dynamic effect or a transition dynamic effect. The splash screen dynamic effect may be a dynamic effect during the startup process before the application enters the user interface, and the transition dynamic effect may refer to the animation effect that appears when switching the interface of the electronic device or switching between applications. For example, the transition animation effect that appears when switching to the main screen of the electronic device, closing an application, etc.

[0187] It can be seen that in the embodiment of the present application, the electronic device can enter the first device state when the target dynamic effect occurs, and when the electronic device is in the first device state, the display screen displays the image at the first refresh rate. It can reduce the number of times the electronic device refreshes the content in the display screen at the highest refresh rate, avoid the electronic device refreshing at the highest refresh rate under unnecessary circumstances, and can reduce the power consumption of the electronic device.

[0188] Furthermore, the target dynamic effect being that the change range of the dynamic effect image exceeds the first change value may include: the target dynamic effect is that the change range of the dynamic effect image within a first preset duration exceeds the first change value. Among them, the first preset duration may be equal to 100 ms, 200 ms, 300 ms, or 500 ms, for example, and the first change value may be 300 pixels or 500 pixels, for example. The embodiment of the present application does not make specific limitations on this.

[0189] Continue to refer to Figure 10 , the image display method provided by the embodiment of the present application may further include S20: When the electronic device is not in the first device state, the display screen displays the image using the refresh rate matching the image layer.

[0190] In the embodiments of the present application, an image layer refers to a layer in a display screen for displaying images. Using a refresh rate that matches the image layer can ensure that the image is smoother and clearer during display, reduce screen tearing or blurring, and provide a good visual experience. If the refresh rate is too high, it will cause the system to consume excessive system resources and electrical energy, and no additional visual effects will be brought. Therefore, using a refresh rate that matches the image layer to display images can also avoid unnecessary resource waste and energy consumption.

[0191] Further, step S20 may include S21: When the electronic device is not in the first device state and the display screen includes M first target layers, the display screen displays the image at a second refresh rate, where the first target layer with a matching frame rate is the active layer, and the matching frame rate of the first target layer is greater than or equal to the first threshold. The active layer includes visible layers whose image content has been updated within the first historical duration. The second refresh rate is determined based on the matching frame rates of the M first target layers.

[0192] The matching frame rate can represent the speed at which the first target layer is updated and rendered per second. Determining the second refresh rate based on the matching frame rate of the first target layer can make the refresh rate of the display screen match the speed at which the first target layer is updated and rendered, and can avoid problems such as tearing of the image content in the first target layer.

[0193] In some implementation manners, the first historical duration may be equal to 1.2 s, and the embodiments of the present application do not make specific limitations thereto.

[0194] It can be understood that the value of M is determined by the actual situation of the layers in the display screen.

[0195] The following introduces the specific steps for determining the second refresh rate based on the matching frame rates of the M first target layers.

[0196] Specifically, the second refresh rate is the maximum refresh rate among refresh rate 1 to refresh rate M, where refresh rate 1 is the desired refresh rate of the first first target layer, refresh rate M is the desired refresh rate of the Mth first target layer, refresh rate 1 is determined based on the matching frame rate of the first first target layer and the optional refresh rates supported by the display screen, and refresh rate M is determined based on the matching frame rate of the Mth first target layer and the optional refresh rates supported by the display screen.

[0197] The step of calculating and then determining the desired refresh rate based on the matching frame rate and the optional refresh rates supported by the display screen may be to calculate the degree of proximity between the matching frame rate and the optional refresh rates, and determine the optional refresh rate closest to the matching frame rate as the desired refresh rate of the first target layer. For example, the method of calculating the degree of proximity is to calculate the multiple relationship between the matching frame rate and the optional refresh rates, and based on the multiple relationship, determine the optional refresh rate closest to the matching frame rate. The method of calculating the degree of proximity may also be to calculate the percentage difference between the matching frame rate and the optional refresh rates. Specifically, the difference between the two can be calculated, and the difference is divided by the matching frame rate, and then multiplied by 100 to obtain a percentage value. The embodiments of the present application do not make specific limitations on this.

[0198] For each first target layer, one or more optional refresh rates can be determined after calculation. For example, if the matching frame rate of the i-th first target layer is 30 Fps, then the desired refresh rate of the i-th first target layer can be 30 Hz and 60 Hz.

[0199] Exemplarily, the optional refresh rates may include 30 Hz, 60 Hz, 90 Hz, and 120 Hz. The display screen may include 3 first target layers. The matching frame rate of the first first target layer may be 30 Fps, and after calculation, its desired refresh rates are determined to be 30 Hz and 60 Hz. The matching frame rate of the second first target layer may be 120 Fps, and after calculation, its desired refresh rate is determined to be 120 Hz. The matching frame rate of the third first target layer may be 60 Fps, and after calculation, its desired refresh rate is determined to be 120 Hz. Then, the second desired refresh rate is the maximum refresh rate among 30 Hz, 60 Hz, and 120 Hz, that is, the second desired refresh rate is 120 Hz.

[0200] The method provided by the embodiments of the present application may further include step S30: when the electronic device is not in the first device state and the display screen does not include the first target layer, the display screen displays an image at a third refresh rate, and the third refresh rate is the default refresh rate of the display screen.

[0201] Among them, the default refresh rate is the refresh rate when no adjustment is made to the refresh rate at which the display screen displays an image. The display screen refreshes the content at the default refresh rate, which can reduce the power consumption of the electronic device and avoid unnecessary resource waste.

[0202] In the embodiments of the present application, the default refresh rate may be 60 Hz. A refresh rate of 60 Hz has wide compatibility in a display screen, and the frame rate of many layers is 60 frames per second. Therefore, refreshing the content in the display screen at 60 Hz as the default refresh rate can provide a good user experience. The default refresh rate may also be determined based on various factors such as the hardware capabilities of the display screen. For example, when the display screen supports a maximum refresh rate of 144 Hz, the default refresh rate may be 90 Hz. The embodiments of the present application do not make specific limitations on this.

[0203] Figure 12 This is the second process schematic diagram of the image display method provided by the embodiments of the present application.

[0204] As Figure 12 shown, the image display method provided by the embodiments of the present application includes the following steps S100 - S500:

[0205] S100: Before the electronic device refreshes the content in the display screen each time, determine whether the electronic device is in the first device state.

[0206] In the embodiments of the present application, the timing for the electronic device to refresh the content in the display screen each time may be controlled by a vertical synchronization signal (VSync). Among them, the vertical synchronization signal may include a VSync - HW signal generated by hardware, as well as a VSync - APP signal and a VSync - SF signal simulated by software. The VSync - HW signal, the VSync - APP signal, and the VSync - SF signal usually arrive periodically at a certain time interval.

[0207] Taking an electronic device equipped with an Android system as an example, when the VSync - APP signal arrives, the SurfaceFlinger process performs the drawing work of one or more layers required for an image frame based on the layer data sent from the foreground application program, and performs layer rendering on the drawn one or more layers; thereafter, when the VSync - SF signal arrives, the SurfaceFlinger process synthesizes the one or more layers that have completed rendering into an image frame, and sends the synthesized image frame to the display screen; thereafter, when the VSync - HW signal arrives, the display screen is refreshed to display the image frame. Therefore, before the electronic device refreshes the content in the display screen each time, it may refer to before the electronic device receives the VSync - HW signal.

[0208] The first device state may include that the electronic device receives a target interaction event or the display screen is playing a target dynamic effect, and the target interaction event includes a click event and / or a first sliding event.

[0209] In the embodiments of the present application, the target interaction event may include a click event (Click) and / or a first sliding event.

[0210] Before introducing the target interaction event, first make some simple introductions to the touch events involved in the electronic device. In an electronic device, a touch display screen is usually used as the main way for input and user interaction. When a user touches or clicks on the display screen, corresponding touch events will be triggered, and these events can include:

[0211] 1. Touch Down event: It means that the user starts to press a finger on the display screen. In this event, the electronic device usually records the position and timestamp where the user presses.

[0212] 2. Touch Move event: It means that the user slides a finger on the display screen. In this event, the electronic device usually records the current position information of the sliding finger, the moving distance, and the timestamp, etc.

[0213] 3. Touch Up event: It means that the user releases the finger and leaves the display screen. In this event, the electronic device usually records the position and timestamp where the finger is released.

[0214] 4. Click event: The Click event is triggered by a click operation. A series of events that occur continuously between the Touch Down event and the Touch Up event are considered a click event. Specifically, when the time difference between the timestamps of the Touch Down event and the Touch Up event on the display screen is less than a certain duration (for example, 300 ms), a Click event will be triggered. It can be seen that the Click event depends on the occurrence of the Touch Down event, and the occurrence of the Touch Down event does not necessarily trigger the Click event. Only when the time difference between the timestamps of the Touch Down event and the Touch Up event is less than a certain duration, the Click event can be triggered.

[0215] It should be added that not any Click event can trigger the first device state. In the embodiments of the present application, it is possible to set a prohibition on triggering the first device state for a specific application. For example, for the Click event of the layer of the input method application, the first device state cannot be triggered.

[0216] The first sliding event can be a sliding event with a sliding speed greater than a first preset threshold. The first sliding event can be triggered depending on a Touch Move event. Specifically, the electronic device can calculate the sliding speed based on the position information of the sliding finger, the moving distance, the timestamp, etc. When the sliding speed is greater than the first preset threshold, the first sliding event can be triggered. It can be understood that when the first sliding event occurs, the content on the display screen will quickly scroll along with the sliding of the user's finger. Therefore, the content on the display screen changes greatly.

[0217] The target animation effect can be an animation effect involving a relatively wide range of changes or rapid changes. Specifically, the target animation effect can be that the change range of the animation image exceeds a first change value. For example, the animation effects involved when an application starts up, the animation effects in games, etc. Specifically, they can be gradient and transition effects, translation and zoom animations, drag and scroll effects, etc. The slight movement of micro icons or the slight swing of graphic elements are non-target animation effects. The specific type of the target animation effect can be determined according to actual needs, and the embodiments of the present application do not make specific limitations on this.

[0218] S200: When the electronic device is in the first device state, refresh the content on the display screen at a first refresh rate.

[0219] Among them, the first refresh rate is the highest refresh rate of the display screen. Or, the first refresh rate is greater than or equal to a preset refresh rate threshold.

[0220] Step S200 is a condition provided by the embodiments of the present application for determining the refresh rate of the displayed image. Based on step S200, the refresh rate of the display image on the display screen can be determined.

[0221] Figure 13 This is the first schematic diagram of the condition for determining the refresh rate of the displayed image in the image display method provided by the embodiments of the present application.

[0222] As Figure 13 shown, the electronic device can refresh the content on the display screen at a first refresh rate when it is in the first device state.

[0223] It can be seen that the image display method provided by the embodiments of the present application does not refresh at the highest refresh rate based on the occurrence of any interaction event, but will switch the display screen to the highest refresh rate only when a click event and / or a first sliding event occurs. In this way, for some relatively simple interaction events or animation effects, the highest refresh rate will not be triggered, and the problem of resource waste caused by frequent refreshing using the highest refresh rate can be solved.

[0224] For example, when a user reads text on a display screen, it usually triggers Touch Down events, Touch Move events, and Touch Up events. The Touch Down event corresponds to the user pressing a finger, the Touch Move event corresponds to the user slowly sliding the page to present new text content on the page, and the Touch Up event corresponds to completing the page turn and releasing the hand. It can be seen that during this process, the changes in the display screen involved are very small, and there is no need to use the highest refresh rate. Based on the embodiments of this application, the above process does not trigger the target interaction event and will not trigger the highest refresh rate.

[0225] The highest refresh rate supported by the display screen of an electronic device is related to various factors such as the hardware capabilities of the electronic device and the display screen technology. The hardware capabilities of the electronic device include the performance of devices such as the processor, graphics processing unit, memory, and storage. The performance of these hardware components determines the computing and graphics processing capabilities of the electronic device. Higher hardware capabilities can support higher refresh rates and smoother display effects. The display screen technology refers to the technology for manufacturing the display screen. For example, a display screen made relying on OLED or AMOLED technology can support a maximum refresh rate of 144Hz.

[0226] In the embodiments of this application, the highest refresh rate can be the maximum value among one or more selectable refresh rates supported by the display screen. For example, if the display screen supports any one of 30Hz, 60Hz, 120Hz, and 144Hz refresh rates, then the electronic device can determine the first refresh rate as the highest refresh rate of 144Hz supported at this time. If the display screen supports any one of 30Hz, 60Hz, and 120Hz refresh rates, then the electronic device can determine the first refresh rate as the highest refresh rate of 120Hz supported at this time.

[0227] In practical applications, the electronic device can dynamically adjust the provided refresh rate range based on its own state. For example, when the electronic device is in the high-performance mode and can support any one of 30Hz, 60Hz, 120Hz, and 144Hz refresh rates, then the electronic device can determine the first refresh rate as 144Hz. When the electronic device is in the power-saving mode and can support any one of 30Hz, 60Hz, and 120Hz refresh rates, then the electronic device can determine the first refresh rate as 120Hz. When the electronic device is in the super power-saving mode and can support 30Hz and 60Hz refresh rates, then the electronic device can determine the first refresh rate as 60Hz.

[0228] It should be noted that, in order not to affect the user experience, the electronic device can also adopt some means to lock the refresh rate. For example, by locking the refresh rate of the display screen, applications or games will be restricted to run within a specific frame rate or frame rate range, and will not exceed or be lower than this frame rate or frame rate range, so as to avoid reducing the game experience due to frequent changes in the refresh rate. If the electronic device locks the refresh rate to 120Hz, then there is only one optional refresh rate at this time. For another example, when the display screen is in a low-brightness display state, the low-brightness display is, for example, a brightness value of 30, and at this time the electronic device can also lock the refresh rate of the display screen, for example, lock it to 30Hz.

[0229] It should be noted that when the nth VSync-HW signal arrives, the steps of refreshing the display screen for the nth time and determining the refresh rate of the display screen can be executed. After that, the arrival time of the (n + 1)th VSync-HW signal is adjusted based on the first refresh rate. For example, when the nth VSync-HW signal arrives at the 0th second, based on steps S100 - S200, the refresh rate of the display screen is determined to be the first refresh rate, that is, 120Hz. At this time, the electronic device can adjust the interval duration between two adjacent VSync-HW signals to 8.33ms, that is, at the 8.33ms, the (n + 1)th VSync-HW signal arrives, and at this time, the steps of refreshing the (n + 1)th time and determining the refresh rate of the display screen can be executed.

[0230] S300: When the electronic device is not in the first device state, determine whether the display screen includes a first target layer.

[0231] Step S300 is another condition different from step S200 for determining the refresh rate of the displayed image. Based on step S300, the refresh rate of the display screen for the displayed image can be determined.

[0232] Figure 14 This is the second schematic diagram of the condition for determining the refresh rate of the displayed image in the image display method provided by the embodiments of the present application.

[0233] As Figure 14 shown, not being in the first device state can also be referred to as the electronic device being in the Non-Boost state.

[0234] In the embodiments of the present application, the layers are divided into three types of layers: the first target layer, the second target layer, and the third target layer. The first target layer is an active layer, and the matching frame rate (desiredFps) of the first target layer is greater than or equal to the first threshold. The active layer includes visible layers whose image content has been updated within the first historical duration. Here, desiredFps is only an exemplary description of the matching frame rate. The first threshold can be equal to 10Hz, for example, and the first historical duration can be equal to 1.2s, for example. The matching frame rate can be preset by the application program specifically, or obtained by statistically analyzing the frequency of the application program or the system sending images to SurfaceFlinger. The specific statistical process will be described in detail below and will not be elaborated here. The matching frame rate represents the update speed of the first target layer. For example, a game application program can set the matching frame rate of the game layer to 60Fps, that is, update 60 times per second.

[0235] The second target layer is the layer that is making an animation effect, and the third target layer is the wallpaper layer, and / or, the layer whose matching frame rate is less than the first threshold, and the status bar layer. It should be added that Figure 14 only exemplarily shows that the first target layer includes Layer1-Layer5, and exemplarily shows the second target layer Layer6. The active layer may also include the third target layer, Figure 14 which is not fully shown.

[0236] In one implementation, the step of determining whether a layer is an active layer may include: before the electronic device refreshes the content of the display screen each time, for all visible layers in the display screen, calculate the difference between the time when the visible layer last updated its image content and the current time. If the difference is less than or equal to the first historical duration, then it can be determined that the visible layer is an active layer.

[0237] The layer updating its image content here may refer to the application program sending layer data to SurfaceFlinger within the first historical duration, or may refer to the attributes of the layer changing within the first historical duration.

[0238] S400: When the display screen includes at least one first target layer, determine at least one desired refresh rate from at least one optional refresh rate respectively based on the matching frame rate of each first target layer, and refresh the content in the display screen at the highest refresh rate among the at least one desired refresh rates.

[0239] Exemplarily, the optional refresh rates may include Rate1, Rate2, Rate3, and Rate4, and Rate1, Rate2, Rate3, and Rate4 may be 30Hz, 60Hz, 90Hz, and 120Hz respectively. For each first target layer, the step of determining at least one desired refresh rate from at least one optional refresh rate may be to score at least one optional refresh rate of the display screen, and determine the optional refresh rate with the highest score as the desired refresh rate of the layer. For example, for layer x, the electronic device scores 30Hz, 60Hz, 90Hz, and 120Hz based on the matching frame rate of layer x, and determines that 60Hz has the highest score, so the desired refresh rate of layer x is 60Hz. Another example is that for layer y, the electronic device scores 30Hz, 60Hz, 90Hz, and 120Hz based on the matching frame rate of layer y, and determines that 120Hz has the highest score, so the desired refresh rate of layer y is 120Hz.

[0240] It can be seen that in the scoring method provided by the embodiments of the present application, for any first target layer, the electronic device can score the optional refresh rate based on the matching frame rate of the first target layer, determine the desired refresh rate of the first target layer, and then select the highest desired refresh rate among the desired refresh rates of all first target layers for refreshing. In this way, in the step of determining the desired refresh rate of the layer, the first target layers do not affect each other, which can make the calculation of the desired refresh rate more accurate. The requirements of each layer are met, and the situation where the dynamic effect experience of the layer with a relatively large desired refresh rate is poor can be avoided.

[0241] Figure 15 This is the first application schematic diagram of the image display method provided by the embodiments of the present application.

[0242] As Figure 15 shown, exemplarily, the user is watching a video in portrait mode while browsing the comment area. At this time, the display screen at least includes layer 4 where the video is being played and comment interface layer 5. The matching frame rate of layer 4 is 60Fps. After scoring, the optional refresh rate with the highest score is 60Hz, that is, the desired refresh rate of layer 4 is 60Hz. The matching frame rate of layer 5 is 30Fps. After scoring, the optional refresh rate with the highest score is 30Hz, that is, the desired refresh rate of layer 5 is 30Hz. The desired refresh rate of layer 4 is greater than that of layer 5, so the electronic device finally refreshes the content on the display screen at 60Hz.

[0243] S500: When the first target layer is not included in the display screen, refresh the content on the display screen at a third refresh rate, and the third refresh rate is the default refresh rate of the display screen.

[0244] In the embodiments of the present application, when there is no first target layer, that is, when the active layer does not include a layer with a matching frame rate greater than or equal to the first threshold, the scoring process is not performed, and the display screen is directly refreshed at the default refresh rate.

[0245] Figure 16 It is a schematic diagram of the display screen provided by the embodiments of the present application that does not include the first target layer.

[0246] The situation where the electronic device is not in the first device state and there is no first target layer may occur, for example, in a scenario where there is an instantaneous dynamic effect on the display screen. As Figure 16 shown in (a), the display screen of the electronic device includes a status bar layer and a layer of the weather application, and each layer has not changed. At this time, the refresh rate is 10 Hz. As Figure 16 shown in (b), a GPS icon appears in the status bar layer. It can be seen that the layer of the weather application has not changed, and an instantaneous dynamic effect has occurred in the status bar layer. At this time, the display screen can be refreshed at the default refresh rate, and the default refresh rate is, for example, 60 Hz.

[0247] In the embodiments of the present application, the default refresh rate can be 60 Hz. It can be understood that the 60 Hz display screen refresh rate, as a centered refresh rate, has wide compatibility, and the 60 Hz refresh rate can provide a relatively smooth and natural animation and video playback experience, can provide good visual effects, and can reduce energy consumption, taking into account both performance and smoothness.

[0248] From the above content, it can be seen that the image display method provided by the embodiments of the present application can use whether the electronic device is in the Boost state as a condition for determining the refresh rate of the displayed image. Whether it is in the Boost state is related to whether a target interaction event or a target dynamic effect occurs. In this way, the highest refresh rate can be applied when the target interaction event or the target dynamic effect arrives, rather than applying the highest refresh rate when any interaction event or dynamic effect arrives, which can avoid wasting resources. When the electronic device is not in the Boost state, the method provided by the embodiments of the present application can also determine whether the display screen includes a first target layer. When the display screen includes a first target layer, the matching frame rate of each first target layer is used to score the optional refresh rate to determine the expected refresh rate of each first target layer. In this way, for any first target layer, the expected refresh rate can be determined based on its own matching frame rate, and the first target layers do not affect each other, which can make the calculation of the expected refresh rate more accurate. Finally, the content on the display screen is refreshed at the highest refresh rate among the expected refresh rates, which can make the refresh effect of the first target layer with the highest expected refresh rate in the best state, and there is no situation such as stuttering. When the display screen does not include the first target layer, the display screen can be refreshed at the default refresh rate.

[0249] In some implementations, before determining whether the display screen includes a first target layer in step S300, it further includes S301: determining whether the display screen includes a second target layer.

[0250] That is to say, the electronic device can first determine whether the display screen includes an active layer that is playing an animation effect.

[0251] Determining whether the display screen includes a first target layer may include S302: in the case where the display screen does not include a second target layer, determining whether the display screen includes a first target layer.

[0252] In this way, in the case where the display screen does not include a second target layer, the refresh rate of the image displayed on the display screen can be determined based on the scoring process.

[0253] In some implementations, in the case where there is a first target layer on the display screen, it can also be determined that the matching frame rates of all the first target layers on the display screen are equal to a first preset value and none of the matching frame rates are preset. If all the first target layers on the display screen are equal to the first preset value and none of the matching frame rates are preset, the display screen is refreshed at the default refresh rate. In the embodiments of the present application, in the case where the active layer does not have a preset matching frame rate, and the number of updates of the active layer within a second historical duration is less than a number threshold and the time intervals of each update are not equal, the matching frame rate of the active layer is the first preset value.

[0254] If at least one of the first target layers on the display screen has a preset matching frame rate, the at least one optional refresh rate of the display screen is scored respectively based on the matching frame rate of each first target layer, and according to the scoring result, at least one desired refresh rate is determined from the at least one optional refresh rate, and the content on the display screen is refreshed at the highest refresh rate among the at least one desired refresh rates.

[0255] Figure 17 This is the third schematic diagram for the conditions used to determine the refresh rate of the displayed image in the image display method provided by the embodiments of the present application.

[0256] In some implementations, as Figure 17 shown, in the embodiments of the present application, after step S301, it may further include S303: in the case where the display screen includes a second target layer, the content on the display screen is refreshed at a first refresh rate.

[0257] That is to say, in the case where the electronic device is not in the first device state, if the display screen is playing an animation effect, the display screen can be refreshed at the highest refresh rate to improve the user experience.

[0258] Exemplarily, as Figure 18As shown in (a) and (b), the display screen at least includes a layer containing content such as text and pictures published by the user in the application. The user can hold down the display screen and slowly pull it down, and then release it. Since the slow pull-down operation is not a click event or the first sliding event, the electronic device will not be triggered to enter the first device state.

[0259] When performing the slow pull-down operation, an update animation effect appears on the display screen. That is to say, an active layer with an animation effect being played appears on the display screen. At this time, the electronic device refreshes the content on the display screen at the first refresh rate, and the first refresh rate can be 120Hz. Next, as Figure 18 shown in (c) and (d), after the user releases the hand, the update animation effect continues to appear on the display screen, that is, the active layer includes the second target layer. At this time, the electronic device continues to refresh at the first refresh rate.

[0260] In this implementation, before the electronic device determines whether the display screen includes the first target layer, it first determines whether the display screen includes the second target layer. If the second target layer is included, the content on the display screen is refreshed at the first refresh rate. If the second target layer is not included, it then determines whether the first target layer is included to perform the scoring step according to the first target layer. In this way, if the electronic device has the first target layer, it can determine the refresh rate of the image displayed on the display screen before performing the scoring process, reducing the calculation amount, increasing the speed of determining the refresh rate, and avoiding resource waste.

[0261] Figure 19 This is the third process schematic diagram of the image display method provided by the embodiments of the present application.

[0262] As Figure 19 shown, in some embodiments, step S100, that is, before the electronic device refreshes the content on the display screen each time, determining whether the electronic device is in the first device state can be implemented through the following steps S601 - S612.

[0263] S601: Before the electronic device refreshes the content on the display screen each time, obtain the cache result. The cache result includes the device state information of the electronic device, the layer state information of the active layer, and the historical refresh rate information of the display screen before the last refresh of the content on the display screen.

[0264] Among them, the screen refresh rate when the electronic device last refreshed the content on the display screen was the first historical refresh rate.

[0265] In the embodiments of the present application, the refresh rate of the image displayed on the display screen can be determined before each refresh of the content on the display screen. After determining the refresh rate, the electronic device can cache the refresh rate as historical refresh rate information, and can also cache the device status information and the layer status information of the active layer for use when determining the refresh rate of the image displayed on the display screen next time.

[0266] In the embodiments of the present application, the device status information may include, in addition to the information of the first device status, the information of other device statuses, such as: Touch status information and IDLE status information.

[0267] Among them, the electronic device can enter the Touch state when receiving a touch event. The touch event can be, for example, a Touch Down event. It can be understood that as long as the user touches the display screen, the electronic device will receive a touch event. The way the user touches the display screen may be clicking on the display screen, long-pressing on the display screen, or swiping on the display screen. Therefore, after the electronic device receives a touch event and enters the Touch state, it may also receive a click event (Click) and / or a first swipe event and enter the first device state from the Touch state.

[0268] In some implementation manners, a screen orientation change can also trigger the electronic device to enter the Touch state.

[0269] The electronic device can enter the IDLE state when all the layers on the display screen are in a static state. All the layers being in a static state can mean that all the layers have not changed during n consecutive refreshes, where n≥2.

[0270] Continue to refer to Figure 14 , the layer status information can include, for example, layer name, desired frame rate, vote type, seamlessness, and weight. In some implementation manners, the layer status can also include focused.

[0271] Among them, the matching frame rate represents the number of times the layer is updated per second. Specifically, it can be preset by the application program or statistically calculated based on the frequency of the application program sending images. For a layer with a preset matching frame rate, the matching frame rate represents the number of frames that the application program expects to render per second for the layer. For example, game developers may expect the game to run at a frame rate of 60 Fps to provide a smooth gaming experience. The display refresh rate is the number of times the display updates its content per second. Therefore, to obtain the best visual effects and smoothness, the matching frame rate should match or be close to the display refresh rate. When the matching frame rate matches the refresh rate, each new image frame of the application layer will be presented when the display is updated, thus ensuring image coherence. If the matching frame rate is lower than the display refresh rate, the image frames may be repeatedly displayed during several consecutive refreshes, resulting in an unsmooth picture.

[0272] The layer types of the first target layer, the second target layer, and the third target layer are different, and the voting types are also different. For example: the voting type of the status bar layer (StatusBar) is NoVote (no voting right). The voting type also includes many other types, such as Min (minimum voting right), Max (maximum voting right), ExplicitDefault (explicit default value), ExplicitExactOrMultiple (explicit exact value or multiple values), Heuristic (statistical type), Default (default voting right), ExplicitExact (explicit exact value), etc. The voting type is also related to the scoring process. The voting types of the layers participating in the scoring are different, and the calculation method for the electronic device to score the optional refresh rate based on the matching frame rate of the layer is also different.

[0273] In the embodiments of the present application, the voting type of the second target layer can be Max, and the voting type of the third target layer can be Min. The voting types of the first target layer with a preset matching frame rate can include ExplicitDefault, ExplicitExactOrMultiple, and ExplicitExact.

[0274] Specifically, ExplicitDefault can mean that the first target layer calls the setFrameRate interface and sets the ANATIVEWINDOW_FRAME_RATE_COMPATIBILITY_DEFAULT parameter.

[0275] ExplicitExactOrMultiple can mean that the first target layer calls the setFrameRate interface and sets the ANATIVEWINDOW_FRAME_RATE_COMPATIBILITY_FIXED_SOURCE parameter.

[0276] ExplicitExact may refer to the first target layer calling the setFrameRate interface and setting the ANATIVEWINDOW_FRAME_RATE_COMPATIBILITY_EXACT parameter.

[0277] The voting type of the first target layer with a matching frame rate of the first preset value is Default. For the first target layer whose matching frame rate is calculated and is greater than or equal to the first threshold, its voting type may be Heuristic.

[0278] Seamlessness refers to whether it is allowed to switch the refresh rate under the condition of affecting the experience, and the seamlessness is generally determined by the application.

[0279] Focus may refer to the layer that is currently responding to user input or operations, or the layer that is displayed in the foreground. Whether a layer is specifically a focus layer can be determined according to the actual situation, and the embodiments of the present application do not make specific limitations in this regard.

[0280] The weight specifically refers to the ratio of the layer size to the display screen size.

[0281] The layer status information can be determined by the application or the operating system to which the layer belongs. When the application submits the layer data and the like of the layer to SurfaceFlinger, the above layer status information can be submitted to SurfaceFlinger together. So that before the electronic device refreshes the content in the display screen each time, it can obtain the layer status information. For the layer information not provided by the application, it can be calculated when determining the layer type, and the specific calculation process will be described in detail below and will not be elaborated here.

[0282] In some embodiments, the electronic device can also save the device status information, the layer status information before each time it refreshes the content in the display screen, and the refresh rate used by the display screen when the electronic device refreshes the content in the display screen each time. The refresh rate used by the display screen when the electronic device refreshes the content in the display screen each time refers to the refresh rate determined before refreshing the display screen. In this way, a cache result can be formed after determining the refresh rate for use when performing step S601 next time.

[0283] Continue to refer to Figure 19 , after step S601, the following steps are further included.

[0284] S602: Determine whether the number of current active layers is the same as the number of active layers before the previous refresh.

[0285] It can be understood that if the number of active layers in the display screen changes when the display screen is refreshed twice, for example, a new layer is added, then when determining the refresh rate of the display screen, the layer state of the newly emerged layer needs to be considered. Therefore, when comparing the cache results in the embodiments of the present application, it is possible to first compare whether the number of active layers before the two refreshes of the display screen has changed. For example, before the nth refresh of the display screen, there are 5 active layers in the display screen in total, and there are also 5 active layers in the display screen before the (n - 1)th refresh of the display screen, and the number of layers has not changed, then the state information of each layer can be further compared. If there are 5 active layers in the display screen in total before the nth refresh of the display screen, and there are 4 active layers in the display screen before the (n - 1)th refresh of the display screen, it means that the cache result is unavailable, and other methods can be adopted to determine the refresh rate of the image displayed on the display screen.

[0286] S603: When the number of active layers is different, determine the number of optional refresh rates.

[0287] If the number of active layers in the display screen before the two refreshes of the display screen is different, it means that active layers may have been added or reduced, then the layer state of the active layers will change. Therefore, the cached historical refresh rate cannot be directly used to refresh the content of the display screen. In this case, it is possible to first determine how many optional refresh rates the display screen has. If there is only one optional refresh rate, then step S607 is executed. If there are multiple optional refresh rates, then step S608 is executed.

[0288] S6041: When the number of active layers is the same, determine whether the layer state information of the current active layers is the same as the layer state information of the active layers before the content of the display screen was refreshed last time.

[0289] S6042: Determine whether the current device state information of the electronic device is the same as the device state information before the content of the display screen was refreshed last time.

[0290] Exemplarily, before the current refresh of the display screen, there are 6 active layers in the display screen, including layer a, layer b, layer c, layer d, layer e, and layer f. The layer states of the 6 active layers can be saved in their respective data structures LayerRequirement in the order of creation of the layers. For example:

[0291] ① name = layer a, vote type = NoVate, desiredFps = nall, Seamlessness = true,

[0292] focused = false, weight = 0.1;

[0293] Among them, name = layer a means that the display screen includes a layer with the layer name layer a, vote type = NoVate means that the vote type of layer layer a is NoVate, desiredFps = nall means that layer layer a has no matching frame rate, Seamlessness = true means that layer layer a allows switching the refresh rate under the condition of affecting the experience, focused = false means that layer layer a is not the focus layer, and weight = 0.1 means that the weight of layer layer a is 0.1.

[0294] ② name = layer b, vote type = Default, desiredFps = 60Fps, Seamlessness = true,

[0295] focused = true, weight = 0.5;

[0296] Among them, name = layer b means that the display screen includes a layer with the layer name layer b, vote type = Default means that the vote type of layer layer b is Default, desiredFps = 60Fps means that the matching frame rate of layer layer b is 60Fps, Seamlessness = true means that layer layer b allows switching the refresh rate under the condition of affecting the experience, focused = true means that layer layer b is the focus layer, and weight = 0.5 means that the weight of layer layer b is 0.5.

[0297] ③ name = layer c, vote type = Default, desiredFps = 60Fps, Seamlessness = true,

[0298] focused = false, weight = 0.4;

[0299] Among them, name = layer c indicates that the display screen includes a layer with the layer name layer c, vote type = Default indicates that the voting type of layer layer c is Default, desiredFps = 60Fps indicates that the matching frame rate of layer layer c is 60Fps, Seamlessness = true indicates that layer layer c allows the refresh rate to be switched when it affects the experience, focused = false indicates that layer layer c is not the focus layer, and weight = 0.4 indicates that the weight of layer layer c is 0.4.

[0300] ④ name = layer d, vote type = NoVate, desiredFps = nall, Seamlessness = true,

[0301] focused = false, weight = 0.1;

[0302] Among them, name = layer d indicates that the display screen includes a layer with the layer name layer d, vote type = NoVate indicates that the voting type of layer layer d is NoVate, desiredFps = nall indicates that layer layer d has no matching frame rate, Seamlessness = true indicates that layer layer d allows the refresh rate to be switched when it affects the experience, focused = false indicates that layer layer d is not the focus layer, and weight = 0.1 indicates that the weight of layer layer d is 0.1.

[0303] ⑤ name = layer e, vote type = Max, desiredFps = 120Fps, Seamlessness = true,

[0304] focused = false, weight = 0.7;

[0305] Among them, name = layer e indicates that the display screen includes a layer with the layer name layer e, vote type = Max indicates that the voting type of layer layer e is Max, desiredFps = 120Fps indicates that the matching frame rate of layer layer e is 120Fps, Seamlessness = true indicates that layer layer e allows the refresh rate to be switched when it affects the experience, focused = false indicates that layer layer e is not the focus layer, and weight = 0.7 indicates that the weight of layer layer e is 0.7.

[0306] ⑥ name = layer f, vote type = Max, desiredFps = 120Fps, Seamlessness = true,

[0307] focused = false, weight = 0.6;

[0308] Among them, name = layer f indicates that the display screen includes a layer with the layer name layer f, vote type = Max indicates that the voting type of layer layer f is Max, desiredFps = 120Fps indicates that the matching frame rate of layer layer f is 120Fps, Seamlessness = true indicates that layer layer f allows the refresh rate to be switched under the condition of affecting the experience, focused = false indicates that layer layer f is not the focus layer, and weight = 0.6 indicates that the weight of layer layer c is 0.6.

[0309] Before the last refresh of the display screen, the display screen included 6 active layers, namely layer a, layer b, layer c, layer d, layer e, and layer g. According to the order of layer creation, the layer status of each layer is as follows:

[0310] ① name = layer a, vote type = NoVate, desiredFps = nall, Seamlessness = true,

[0311] focused = false, weight = 0.1;

[0312] ② name = layer b, vote type = Default, desiredFps = 60Fps, Seamlessness = true,

[0313] focused = true, weight = 0.5;

[0314] ③ name = layer c, vote type = Default, desiredFps = 60Fps, Seamlessness = true,

[0315] focused = false, weight = 0.4;

[0316] ④name = layer d, vote type = NoVate, desiredFps = nall, Seamlessness = true,

[0317] focused = false, weight = 0.1;

[0318] ⑤name = layer e, vote type = Max, desiredFps = 120Fps, Seamlessness = true,

[0319] focused = false, weight = 0.7;

[0320] ⑥name = layer g, vote type = Default, desiredFps = 60Fps, Seamlessness = true,

[0321] focused = false, weight = 0.4;

[0322] Among them, name = layer g means that the display screen includes a layer with the layer name layer g, vote type = Default means that the voting type of layer layer g is Default, desiredFps = 60Fps means that the matching frame rate of layer layer f is 60Fps, Seamlessness = true means that layer layer g allows the refresh rate to be switched under the condition of affecting the experience, focused = false means that layer layer g is not the focus layer, and weight = 0.4 means that the weight of layer layer f is 0.4.

[0323] In step S602, according to the order of creation of the layers, traverse the active layers layer a, layer b, layer c, layer d, layer e, and layer f before the current display screen is refreshed, and determine whether the layer status information of the active layer has changed compared with before the previous display screen was refreshed. Among them, any change in name, vote type, desiredFps, Seamlessness, focused, or weight means that the layer status information has changed. Exemplarily, by comparison, it can be seen that the name, voting type, matching frame rate, and weight of the sixth layer before the current display screen is refreshed have all changed.

[0324] Further, in the embodiments of the present application, it is also possible to compare whether the device status information of the electronic device has changed. For example, before the current display screen is refreshed, the device status is the Touch state, and before the previous display screen was refreshed, the device status was the IDLE state, indicating that the device status information of the display screen has changed compared with that before the previous refresh. Another example is that before this refresh of the display screen, the device status is the Touch state, and before the previous refresh of the display screen, the device status is also the Touch state, indicating that the device status information has not changed compared with the previous refresh of the display screen.

[0325] It should be noted that determining whether the layer status information of the current active layer is the same as that of the active layer before the previous refresh, and determining whether the device status information of the current electronic device is the same as that of the electronic device before the previous refresh can be carried out simultaneously or sequentially. The embodiments of the present application do not make specific limitations on this.

[0326] S605: When the layer status information of the active layer is the same and the device status information of the electronic device is the same, refresh the content in the display screen at the first historical refresh rate.

[0327] The number of active layers is the same, the layer status information of the active layers is the same, and the device status information of the electronic device is the same, indicating that the cache result is available. Therefore, the content in the display screen can be refreshed at the first historical refresh rate of the previous refresh.

[0328] S606: When the layer status information of the active layer is different or the device status information of the electronic device is different, determine the number of optional refresh rates.

[0329] If the number of active layers is the same, but the layer status information of the active layers is different or the device status information of the electronic device is different, it indicates that the layer status of the layers in the display screen has changed, or the device status of the electronic device has changed. Therefore, the cached first historical refresh rate cannot be directly used to refresh the content of the display screen. In this case, it is also possible to determine how many optional refresh rates the display screen has. If there is only one optional refresh rate, then step S607 is executed. If there are multiple optional refresh rates, then step S608 is executed.

[0330] Step S607, when there is only one optional refresh rate, refresh the content in the display screen at the optional refresh rate.

[0331] It is understandable that the electronic device can dynamically adjust the specific value of the optional refresh rate, for example, adjust the optional refresh rate based on the power supply status (high performance, power saving, super power saving), lock the display refresh rate based on application requirements or system animations, or lock the refresh rate based on the display brightness, etc. In the scenario of locking the refresh rate, the electronic device only includes one optional refresh rate.

[0332] Therefore, the embodiment of the present application can first determine whether only one optional refresh rate is included. If only one optional refresh rate is included, then the optional refresh rate can be used to refresh the content in the display screen to reduce the amount of calculation.

[0333] S608: In the case of including multiple optional refresh rates, determine whether each layer in the display screen is in a static state and the electronic device does not receive a touch event.

[0334] Among them, each layer is in a static state, indicating that the electronic device is in an IDLE state, and the display screen does not receive a touch event, indicating that the electronic device is not in a Touch state.

[0335] S609: When all layers in the display screen are in a static state and the electronic device does not receive a touch event, refresh the content in the display screen at a lowest refresh rate among multiple optional refresh rates.

[0336] Figure 20 This is a fourth schematic diagram of the conditions for determining the refresh rate of a displayed image in the image display method provided in an embodiment of the present application.

[0337] like Figure 20 As shown, when the electronic device is in the IDLE state and not in the Touch state, the lowest refresh rate can be determined from multiple optional refresh rates, and the content in the display screen can be refreshed at the lowest refresh rate. It is understandable that when all layers in the display screen of the electronic device are in a static state and no touch event occurs, the content in the display screen will not change, so a lower display screen refresh rate can be set.

[0338] Take the lowest refresh rate of 10Hz as an example. Figure 21 exemplarily shows that the display screen includes a status bar layer, a wallpaper layer, and a launcher layer. At this time, the user does not touch the display screen, and the status bar layer, the wallpaper layer, and the launcher layer are all in a static state, and the refresh rate of the electronic device refreshing the display screen is the minimum refresh rate of 10Hz.

[0339] S610: When at least one layer in the display screen is in a non-static state or the display screen receives a touch event, determine whether the active layer includes only the third target layer.

[0340] Among them, the third target layer includes a wallpaper layer, a layer with a matching frame rate less than the first threshold, and a status bar layer.

[0341] Among them, at least one layer is in a non - stationary state, indicating that the electronic device is not in the IDLE state, and the display screen receives a touch event, indicating that the electronic device is in the Touch state. The electronic device being in the Touch state or not in the IDLE state indicates that the next refresh process may involve changes in the layers. Therefore, the refresh rate can be determined based on the actual conditions of each layer.

[0342] At least one layer in the display screen being in a non - stationary state or the display screen receiving a touch event specifically includes three actual situations. Situation one: The electronic device is in the Touch state and in the IDLE state; Situation two: The electronic device is in the Touch state and not in the IDLE state; Situation three: The electronic device is not in the Touch state and not in the IDLE state.

[0343] Furthermore, Figure 22 (a) and (b) in [reference] exemplarily show the situation where there is a layer in a non - stationary state and the display screen receives a touch event. Among them, when the user's finger slowly slides up on the non - responsive area of the display screen, the electronic device receives a Touch Down event and enters the Touch state. The non - responsive area means that the electronic device does not respond to any Touch Down event received in this area. At this time, the status bar layer and the clock layer will change and enter the non - stationary state.

[0344] Taking the lowest refresh rate among the optional refresh rates as 10Hz as an example, continue to refer to Figure 22 (b) in [reference]. In the case where a Touch Down event occurs and the status bar layer and the clock layer are in a non - stationary state, the active layers include the status bar layer and the clock layer, and the launcher layer is not an active layer. It can be understood that the status bar layer is the third target layer, and the matching frame rate of the clock layer is 1Hz, that is, it is updated once per second, and the matching frame rate is less than the first threshold of 10Hz. The clock layer is also the third target layer. Thus, Figure 22 the active layers in (b) in [reference] only include the third target layers. At this time, the electronic device can refresh the content in the display screen at the lowest refresh rate of 10Hz among the optional refresh rates.

[0345] S611: In the case where the active layers only include the third target layers, refresh the content in the display screen at the lowest refresh rate among the multiple optional refresh rates.

[0346] Figure 23 This is the fifth schematic diagram for determining the conditions of the refresh rate of the displayed image in the image display method provided by the embodiments of the present application.

[0347] As Figure 23 shown, if all active layers only include the third target layer, it indicates that the changes involved in the layers on the display screen are relatively small. Then, a lower display screen refresh rate can be adopted. Therefore, the content on the display screen can be refreshed at the lowest refresh rate among multiple selectable refresh rates. Figure 23 It is shown in that the third target layer may include a wallpaper layer and a layer with a matching frame rate less than the first threshold, denoted as layer7. The third target layer may also include a status bar layer, denoted as layer8.

[0348] S612: When the active layer includes the third target layer and also includes other layers, determine whether the electronic device is in the first device state.

[0349] It can be understood that if all active layers include other layers in addition to the third target layer, it can indicate that there are active layers with relatively large changes on the display screen. Therefore, it can be further determined whether the electronic device is in the first device state, and the refresh rate of the image displayed on the display screen is determined based on the device state.

[0350] Figure 24 This is a schematic flowchart of the process for comparing with the cache result provided by the embodiments of the present application.

[0351] As Figure 24 shown, at 0ms, the electronic device performs the (n - 1)th refresh. Here, the refresh rate of the (n - 1)th refresh is not limited. After the (n - 1)th refresh ends, the electronic device can perform the step of determining the refresh rate of the image displayed on the display screen. Here, the process of comparing the cache result is not shown, but the refresh rate of the nth refresh of the display screen is directly determined by steps S606 - S612. The final 120Hz is a schematic value. It should be noted that the device state information and layer state information applied to steps S606 - S612 are determined based on the state after the (n - 1)th refresh ends. Then, the electronic device can cache the device state information, layer state information, and the refresh rate of 120Hz. Since the refresh rate of the display screen determined before the nth refresh is 120Hz, the nth refresh can be performed when 8.33ms arrives.

[0352] Further, after the nth refresh ends, the electronic device can execute steps S601 - S612 again to determine the refresh rate of the image displayed on the display screen. During the process of determining the refresh rate this time, the electronic device first obtains the cache result before the nth refresh to obtain the device state information, layer state information, and the first historical refresh rate of 120 Hz before the nth refresh. Here, it is assumed that the device state information and layer state information after the nth refresh (before the (n + 1)th refresh) have not changed compared to the device state information and layer state information in the cache result. Therefore, the refresh can continue at 120 Hz, that is, the (n + 1)th refresh is executed when 16.66 ms arrives.

[0353] Further, after the (n + 1)th refresh ends, the electronic device can execute steps S601 - S612 again to determine the refresh rate of the image displayed on the display screen to determine the refresh rate of the (n + 2)th refresh of the display screen.

[0354] The steps for determining the layer type are introduced below with reference to the accompanying drawings.

[0355] Figure 25 It is a flowchart for determining the layer type provided by the embodiment of the present application.

[0356] Figure 26 It is Figure 25 an example diagram of the shown process.

[0357] As Figure 25 and Figure 26 shown, the electronic device can execute the following steps S701 - S707 before each refresh of the display screen to determine the layer type of each active layer in the display screen. In practical applications, the layer type of each active layer in the display screen can be determined after the nth refresh of the display screen and before the (n + 1)th refresh of the display screen, and steps S601 - S612 are executed based on the layer type. The active layer for which the layer type is determined can be the layer obtained by removing duplicates from the active layer for which the layer type was determined after the (n - 1)th refresh of the display screen.

[0358] S701: For the active layer in the display screen, determine whether the active layer has called the target interface.

[0359] Among them, the target interface can be the setFrameRate interface for setting the matching frame rate. If the active layer has called the target interface, it means that the active layer contains matching frame rate information, and the matching frame rate information is preset by the application program. If the active layer has not called the target interface, it means that the active layer does not contain matching frame rate information. The matching frame rate information is specifically used to indicate the matching frame rate of the active layer.

[0360] Whether the active layer calls the target interface is determined by the application to which the active layer belongs.

[0361] S702: When the active layer has not called the target interface, determine whether the active layer is the status bar layer.

[0362] S703: When the active layer is not the status bar layer, determine whether the active layer is the wallpaper layer. When the active layer is the wallpaper layer, end the calculation.

[0363] S704: When the active layer is not the wallpaper layer, determine whether the active layer is and is a layer playing an animation effect; among them, the basis for determining whether the active layer is playing an animation effect can be that there are changes such as element scaling, rotation, fade-in and fade-out in two or more consecutive frames of the layer. If the elements of the layer involve the above changes in two or more consecutive frames, then the layer is playing an animation effect.

[0364] S7051: When the active layer is a layer playing an animation effect, determine the active layer as the second target layer.

[0365] S7052: When the active layer is not a layer playing an animation effect, calculate the matching frame rate of the active layer.

[0366] S706: Determine the active layer with a matching frame rate less than the first threshold as the third target layer, and determine the active layer with a matching frame rate greater than or equal to the first threshold as the first target layer.

[0367] Specifically, the steps of calculating the matching frame rate of the active layer in step S7052 specifically include S7052a: When the active layer does not contain matching frame rate information, determine the matching frame rate based on the number of updates and the update time interval of the active layer within the second historical duration.

[0368] Among them, the second historical duration can be 1s, 2s, 5s or 10s. The number of updates can refer to the number of times the application submits the graphic object of the layer to SurfaceFlinger. The update time interval can refer to the time interval between each submission. For example, for the second hand layer of the clock application, the clock application submits a new graphic object of the second hand layer to SurfaceFlinger every second. Therefore, within 5s, the number of updates of the second hand layer is 5 times, and the update time interval is 1s.

[0369] Further, step S7052a includes the following steps S7052a-1 and S7052a-2.

[0370] S7052a-1: When the number of updates is greater than or equal to the threshold number of times or the time intervals for each update are equal, the matching frame rate is the ratio of the number of updates to the second historical duration.

[0371] Among them, the matching frame rate calculated based on a relatively small number of updates has low accuracy. Moreover, if the differences between the respective update time intervals are large, it means that the application does not send images to this layer regularly, then the calculated matching frame rate cannot represent the actual update status of the layer and the demand for the refresh rate. Therefore, the embodiments of the present application can calculate the matching frame rate for active layers where the number of updates is greater than or equal to the threshold number of times or the time intervals for each update are equal. The threshold number of times is, for example, equal to 3 times, and the second historical duration can be, for example, equal to 3s or 5s. The embodiments of the present application do not make specific limitations on this.

[0372] Exemplarily, when the second historical duration is 5s, for the second hand layer of the clock application, the ratio of the number of updates to the second historical duration = 5 / 5 = 1 time / second. Then the matching frame rate of the second hand layer of the clock application is 1fps.

[0373] S7052a-2: When the number of updates is less than the threshold number of times and the time intervals for each update are not equal, the matching frame rate is a first preset value.

[0374] Among them, the first preset value is, for example, 60Fps. In this way, for layers where the number of updates is less than the threshold number of times and the time intervals for each update are not equal and for which the matching frame rate cannot be calculated, their matching frame rates can be determined to be 60Fps, avoiding setting the matching frame rate to the highest refresh rate among the optional refresh rates, and thus avoiding the electronic device refreshing the display screen at the highest refresh rate, resulting in waste of resources.

[0375] It can be understood that for some layer status information not provided by the application to SurfaceFlinger, such as voting type information and matching frame rate information, it can be determined when calculating the layer type.

[0376] S707: Determine the active layer with the matching frame rate being the first preset value as the first target layer.

[0377] In some implementation manners, after step S701, it further includes:

[0378] S708: When the target interface has been called by the active layer, determine the active layer that has called the target interface as the first target layer.

[0379] It can be understood that the active layer calling the target interface means that the active layer has a preset matching frame rate.

[0380] Figure 27A flowchart for scoring optional refresh rates provided by embodiments of the present application.

[0381] Figure 28 For Figure 27 An example diagram of the shown process.

[0382] The following specifically describes step S400 with reference to the accompanying drawings, that is, in the case where the display screen includes at least one first target layer, the step of determining at least one desired refresh rate from at least one optional refresh rate.

[0383] Among them, in step S400, for the first target layer with a voting type of ExplicitExact, its desired refresh rate can be set equal to the matching frame rate.

[0384] For the first target layer with voting types of Heuristic, ExplicitDefault, ExplicitExactOrMultiple, and Default, step S400 may specifically include the following steps:

[0385] S401: For each first target layer, when the matching frame rate of the first target layer is less than or equal to the second threshold, the optional refresh rates less than or equal to the second threshold are determined as the candidate refresh rates of the first target layer.

[0386] Among them, the second threshold is greater than the first threshold. Exemplarily, the second threshold can be 60Fps. The optional refresh rates can include, for example, 30Hz, 60Hz, 90Hz, 120Hz, and 144Hz. If the matching frame rate of the first target layer is 60Fps, then the candidate refresh rates of this first target layer are 30Hz and 60Hz. In this way, only 30Hz and 60Hz need to be scored to determine the desired refresh rate, without the need to score 90Hz, 120Hz, and 144Hz, thereby reducing the calculation amount.

[0387] S402: For each first target layer, when the matching frame rate is greater than the second threshold, the optional refresh rates greater than the second threshold are determined as the candidate refresh rates.

[0388] Exemplarily, if the matching frame rate of the first target layer is equal to 90Fps, then the first target layer can score 90Hz, 120Hz, and 144Hz, without the need to score 30Hz and 60Hz.

[0389] In some implementation manners, the electronic device can also score each optional refresh rate based on the matching frame rate of the first target layer to obtain the desired refresh rate of the first target layer. Embodiments of the present application do not make specific limitations on this.

[0390] S403: Score each candidate refresh rate based on the matching frame rate.

[0391] S404: Determine the refresh rate with the highest score as the desired refresh rate of the first target layer.

[0392] In this way, not only can the desired refresh rate close to the matching frame rate of the layer be selected, but also the calculation amount can be reduced and the speed of determining the refresh rate can be improved.

[0393] Next, step S403, that is, the step of scoring each candidate refresh rate based on the matching frame rate, will be further explained.

[0394] In some implementation manners, step S403 may specifically include the following steps S4031 - S4032:

[0395] S4031: When the candidate refresh rate is equal to the matching frame rate, determine the candidate refresh rate as the first score.

[0396] Wherein, if the candidate refresh rate displayRate = the matching frame rate desiredFps, then the candidate refresh rate can be determined as the first score, and the first score is, for example, 1.

[0397] Exemplarily, desiredFps = 30Fps, displayRate includes 30Hz and 60Hz, 30Hz = 30Fps, and the score of 30Hz is the first score, that is, 1 point.

[0398] It should be added that in the embodiments of the present application, the desired refresh rate of the first target layer can be one or more, and the specific quantity can be determined by the calculation result.

[0399] S4032: When the candidate refresh rate is not equal to the matching frame rate, determine the candidate refresh rate as the second score, and the second score is less than the first score.

[0400] Exemplarily, 60Hz ≠ 30Fps, and the score of 60Hz is the second score, and the second score can be obtained based on calculation, and the specific calculation process will be described in detail below.

[0401] In this way, the candidate refresh rate with the first score can be the desired refresh rate. There is no difference between the desired refresh rate and the matching frame rate, and the matching frame rate of the layer is satisfied, which can make the display effect of the layer smooth and improve the user experience.

[0402] In some other implementation manners, step S403 may specifically further include the following steps S4033 - S4034:

[0403] S4033: When the candidate refresh rate is an integer multiple of the matching frame rate, determine the candidate refresh rate as the first score.

[0404] Where displayRate = (N * desiredFps), the score of the optional refresh rate is determined as the first score, and the first score is, for example, 1.

[0405] Exemplarily, desiredFps = 30Fps, displayRate includes 30Hz and 60Hz, 30Hz = (1 * 30Fps), the score of 30Hz is the first score, that is, 1 point. 60Hz = (2 * 30Fps), and the score of 60Hz is also the first score, that is, 1 point.

[0406] S4034: When the candidate refresh rate is not an integer multiple of the matching frame rate, determine the candidate refresh rate as the second score.

[0407] Among them, the situation that the candidate refresh rate is not an integer multiple of the matching frame rate includes at least the following two cases: the candidate refresh rate is greater than the matching frame rate (displayRate > desiredFps) and the candidate refresh rate is less than the matching frame rate (displayRate < desiredFps). In these two cases, the second score of the candidate refresh rate is less than the first score. When the first score is 1, the score of the candidate refresh rate is a number less than 1. The specific scoring process is shown in the following table:

[0408] Table 1 is the formula for scoring the optional refresh rate

[0409]

[0410]

[0411] Specifically, for the first target layer with the voting type of ExplicitDefault, when the candidate refresh rate is less than the matching frame rate, the formula score = displayRate / desiredFps can be used to calculate the second score. When the candidate refresh rate is greater than the matching frame rate, the formula score = displayRate / (multiplier * desiredFps) can be used to calculate the second score, where multiplier is a multiple, for example, it can be equal to the least common multiple of displayRate and desiredFps, and the specific value can be determined according to the actual situation. It should be noted that the second score calculated based on the formula score = displayRate / (multiplier * desiredFps) is less than 1.

[0412] For the first target layer with voting types of Default, Heuristic, and ExplicitExactOrMultiple, when the candidate refresh rate is an approximate integer multiple of the matching frame rate (isFractionPairOrMultiple), the score of the candidate refresh rate can be set to 0.8, that is, the second score is equal to 0.8. When the candidate refresh rate is less than the matching frame rate, the second score can be calculated using the formula score = displayRate / (desiredFps * (10 + 1)). When the candidate refresh rate is greater than the matching frame rate, the second score can be calculated using the formula score = 1 / iter + 2, where iter represents the number of iterations, and the specific value can be determined according to the actual situation, and the embodiments of the present application do not make specific limitations on this. It should be noted that the second score calculated based on the formula score = 1 / iter + 2 is a value between 0.1 and 0.5.

[0413] It can be seen that when scoring each candidate refresh rate based on the matching frame rate in the embodiments of the present application, the step of calculating the weight weight is not involved, that is, the layer size does not affect the calculation of the desired refresh rate. In this way, both the layer with a small weight and the layer with a large weight can obtain a desired refresh rate close to the matching frame rate, which can avoid the large layer with a low refresh rate affecting the selection of the desired refresh rate of the small layer with a high refresh rate, and further avoid affecting the animation experience of the small layer with a high refresh rate by the large layer with a low refresh rate.

[0414] It should be noted that since the matching frame rate of the first target layer with the voting type of Default is the first preset value, rather than the maximum value supported by the display screen or the highest refresh rate among the selectable refresh rates, it is possible to avoid the refresh rate of the image displayed on the display screen being too large when the display screen includes this layer, thereby avoiding resource waste.

[0415] The device state of the electronic device will be introduced below with reference to the accompanying drawings.

[0416] Figure 29 This is a schematic diagram of the electronic device switching the device state provided by the embodiments of the present application.

[0417] As Figure 29 shown in (a) below, the embodiments of the present application set a first timer for the first device state. Specifically, the electronic device can switch between the first device state (Boost state) and the non-first device state (Non-Boost state).

[0418] S801: When the electronic device receives a target interaction event or the display screen is playing a target animation, it enters the first device state and starts the first timer, and the first timer is set with a first timing duration.

[0419] Among them, the target interaction event is, for example, a click event or a first sliding event, and the target animation effect is, for example, the application startup screen animation effect, etc. For specific details, reference can be made to the foregoing content, which will not be elaborated here.

[0420] In some implementation manners, when a click (Click) event occurs, the electronic device can start a first timer. In other implementation manners, when the sliding speed corresponding to a Touch Move event is greater than a certain threshold, it is considered that a first sliding event occurs, and the electronic device can start a first timer. When the target animation effect appears on the display screen, or when the graphic object of the layer received by SurfaceFlinger corresponds to the target animation effect, it can be considered that the target animation effect occurs, and the electronic device can start a first timer.

[0421] It should be noted that the start of the timing of the first timer corresponds to the electronic device entering the first device state, the first timer being in the timing process corresponds to the electronic device being in the first device state, and the timing of the first timer exceeding the first timing duration corresponds to the electronic device exiting the first device state.

[0422] In some implementation manners, the first timing duration can be equal to 200 ms, for example, and the embodiments of the present application do not make specific limitations on this.

[0423] S802: When the timing of the first timer exceeds the first timing duration, and / or when the electronic device receives a second sliding event, and / or when all the layers on the display screen are in a static state, and / or when the target animation effect ends, the electronic device exits the first device state. The second sliding event can be a sliding event with a sliding speed less than or equal to a first preset threshold.

[0424] It can be understood that whenever a target interaction event occurs or a target animation effect appears, the electronic device can start the first timer. Therefore, when the timing of the first timer exceeds the first timing duration, it means that no new target interaction event occurs or target animation effect appears within the first timing duration, and then the first device state can be exited to save resources.

[0425] The electronic device receiving the second sliding event can specifically refer to: after the user presses a finger to perform a fast sliding operation to trigger the first device state, the user slows down the sliding operation speed, and the fast sliding operation changes to a slow sliding operation. The fast sliding operation and the slow sliding operation are the user's single press-and-hold operation. It can be understood that the key factor for the first sliding event to trigger the first device state is the fast sliding operation speed. Therefore, when the sliding speed decreases, the first sliding event switches to the second sliding event, and the layer change involved in the second sliding event is generally less than the layer change involved in the first sliding event. Therefore, in the embodiments of the present application, the second sliding event will interrupt the first device state.

[0426] In some implementations, when the electronic device enters the IDLE state, it also interrupts the first device state. Specifically, after the last re-timing of the first timer, click events that occur do not trigger changes in the layer content. For example, if the click operation is performed by the user on a non-responsive area of the display screen, and the non-responsive area is, for example, a static wallpaper, then the click events do not trigger changes in the layer. At this time, the electronic device may enter the IDLE state, and after the electronic device enters the IDLE state, the first device state is interrupted.

[0427] In some implementations, the end of the target animation effect means that the layer may no longer change frequently, so it also interrupts the first device state of the electronic device.

[0428] It can be seen that in the embodiments of the present application, to determine whether the electronic device is in the first device state, it can be determined based on the timing duration of the first timer. If the timing of the first timer does not exceed the first timing duration, then the electronic device is in the first device state. If the timing of the first timer exceeds the first timing duration, then the electronic device is not in the first device state, that is, it is in a non-first device state.

[0429] In the embodiments of the present application, an interface can be provided externally. The interface can define target interaction events, target animation effects, etc. for the application program to call. When the application program calls the interface, if the user operates on the layer of the application program with a target interaction event or a target animation effect appears on the layer of the application program, the first device state can be triggered. The interface can also define access permissions to prohibit the call of, for example, the input method application program. The embodiments of the present application do not make specific limitations on this.

[0430] As Figure 29 shown in (b) in , in the embodiments of the present application, the electronic device can switch between the Touch state and the Non-Touch state.

[0431] S803: When the display screen receives a touch event and / or the touch event lasts for more than a second preset duration, the electronic device enters the Touch state and starts a second timer. The second timer is set with a second timing duration.

[0432] Among them, the second timing duration should be greater than the second preset duration. The second preset duration can be, for example, equal to 50 ms or 100 ms. The embodiments of the present application do not make specific limitations on this. The touch event is, for example, a Touch Down event or a Touch Move event. The specific explanations of the Touch Down event and the Touch Move event can be seen in the foregoing content and will not be elaborated here.

[0433] In practical applications, when a Touch Down event occurs, the electronic device can start a second timer. During the period when the timing of the second timer does not exceed the second timing duration, if a Touch Move event occurs and the Touch Move event lasts for a certain duration, then the electronic device can restart the second timer, that is to say, the second timer will start timing again. For example, as the Touch Move event occurs, the electronic device is notified every 100 ms, so that the electronic device can restart the second timer. Therefore, whenever a Touch Down event occurs, the electronic device can restart the second timer, and whenever the Touch Move event lasts for 100 ms, the electronic device can restart the second timer.

[0434] In some implementation manners, the second timing duration can be equal to 200 ms, 500 ms, 1000 ms, and the embodiments of the present application do not make specific limitations thereto.

[0435] S804: When the timing of the second timer exceeds the second timing duration, the electronic device exits the Touch state.

[0436] When the timing of the second timer exceeds the second timing duration, it indicates that no touch event occurs or the touch event stops lasting within the second preset duration, then the Touch state can be exited.

[0437] In the embodiments of the present application, an interface can be provided externally, and touch events and the like can be defined in the interface for application programs to call. When an application program calls the interface, if the user operates a touch event on the layer of the application program, the Touch state can be triggered.

[0438] As Figure 29 shown in (c) herein, in the embodiments of the present application, the electronic device can switch between the IDLE state and the Non-IDLE state.

[0439] S805: When a non-static layer appears on the display screen, the electronic device exits the IDLE state and starts a third timer, and the third timer is set with a third timing duration.

[0440] Wherein, if the content or attribute of the layer changes, etc., the layer can be considered to be in a non-static state. The attributes of the layer are, for example, the transparency, position, size, style or transition effect of the layer, etc., and the embodiments of the present application do not make specific limitations thereto. When a non-static layer appears on the display screen, the third timer is reset and starts timing, and the electronic device enters the Non-IDLE state.

[0441] In some implementation manners, the third timing duration can be equal to 200 ms, and the embodiments of the present application do not make specific limitations thereto.

[0442] S806: When the timing of the third timer exceeds the third timing duration, the electronic device enters the IDLE state.

[0443] Wherein, whenever a non-static layer appears on the display screen, the electronic device can start the third timer. Therefore, when the timing of the third timer exceeds the third timing duration, it indicates that no new non-static layer has appeared within the third timing duration. Thus, the electronic device can exit the Non-IDLE state and enter the IDLE state.

[0444] It should be supplemented that after the first timer is reset and timed for the last time, the click events that occur and do not trigger changes in the layer content do not necessarily cause the electronic device to enter the IDLE state. This is because, if the click events do not trigger changes in the layer content, but the third timer is in a state where its timing has not exceeded the third timing duration, then the electronic device is still in the Non-IDLE state. After the timing of the third timer exceeds the third timing duration, the electronic device can enter the IDLE state, and only then will the first device state be interrupted.

[0445] Figure 30 It is a schematic structural diagram of an image display device provided by an embodiment of the present application.

[0446] As Figure 30 shown, in one embodiment, the electronic device can implement corresponding functions through the Figure 30 hardware device shown. The device may include: a touch screen 901, a memory 902, a processor 903, and a communication module 904. The above-mentioned components can be connected through one or more communication buses 905. The touch screen 901 may include a display panel 9011 and a touch sensor 9012. Among them, the display panel 9011 is used to display images, and the touch sensor 9012 can transmit the detected touch operations to the application processor 903 to determine the type of touch event and provide visual output related to the touch operation through the display panel 9011. The processor 903 may include one or more processing units. For example: the processor 903 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor, etc. Among them, different processing units may be independent devices or integrated in one or more processors. The memory 902 is coupled to the processor 903 and is used to store various software programs and / or computer instructions. The memory 902 may include volatile memory and / or non-volatile memory. When the processor executes the computer instructions, the electronic device can execute each function or step of the above method embodiment.

[0447] When the software program and / or multiple sets of instructions in the memory 902 are executed by the processor 903, the electronic device implements the following method steps: when the electronic device is in the first device state, determine the first refresh rate, where the first refresh rate is the highest refresh rate of the display screen of the electronic device; the trigger conditions for the electronic device to enter the first device state include: the electronic device receives a first interaction event, or a target animation effect appears on the display screen; use the first refresh rate to refresh the content displayed on the display screen of the electronic device.

[0448] This application also provides an electronic device, including: a processor, a memory, and a touch screen; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image display method in any implementation manner in the above embodiments.

[0449] This application embodiment also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices. Exemplarily, the interface circuit can read the instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and this application embodiment does not make specific limitations in this regard.

[0450] This application embodiment also provides a computer-readable storage medium, which includes computer instructions, and when the computer instructions run on the above electronic device, the electronic device executes the various functions or steps executed in the above method embodiments.

[0451] This application embodiment also provides a computer program product, and when the computer program product runs on a computer, the computer executes the various functions or steps executed in the above method embodiments.

[0452] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0453] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0454] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0455] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0456] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0457] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image display method, which is applied to an electronic device, Characterized in that, Comprising: When the electronic device is in the first device state, the display screen of the electronic device displays an image at a first refresh rate, where being in the first device state of the electronic device includes: the electronic device receives a target interaction event, and the target interaction event includes: a click event and a first sliding event, and the first sliding event is a sliding event with a sliding speed greater than a first preset threshold; the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold.

2. The image display method according to claim 1, Characterized in that, The electronic device being in the first device state further includes: the display screen is playing a target dynamic effect, where the target dynamic effect is that the change range of the dynamic effect image exceeds a first change value.

3. The image display method according to claim 2, Characterized in that, The target dynamic effect being that the change range of the dynamic effect image exceeds a first change value includes: the target dynamic effect is that the change range of the dynamic effect image exceeds the first change value within a first preset duration.

4. The image display method according to any one of claims 1-3, Characterized in that, Further comprising: When the electronic device is not in the first device state, the display screen displays an image using a refresh rate matching the image layer.

5. The image display method according to claim 4, Characterized in that, When the electronic device is not in the first device state, the display screen displays an image using a refresh rate matching the image layer, including: When the electronic device is not in the first device state and the display screen includes M first target layers, the display screen displays an image at a second refresh rate, where the first target layer is an active layer, and the matching frame rate of the first target layer is greater than or equal to a first threshold, and the active layer includes a visible layer whose image content has been updated within a first historical duration, and the second refresh rate is determined based on the matching frame rates of the M first target layers; When the electronic device is not in the first device state and the display screen does not include the first target layer, the display screen displays an image at a third refresh rate, and the third refresh rate is the default refresh rate of the display screen.

6. The image display method according to claim 5, Characterized in that, The second refresh rate is the maximum refresh rate among refresh rate 1 to refresh rate M, where refresh rate 1 is the refresh rate expected by the first of the first target layers, and refresh rate M is the refresh rate expected by the Mth of the first target layers; Refresh rate 1 is determined based on the matching frame rate of the first of the first target layers and the optional refresh rates supported by the display screen, and refresh rate M is determined based on the matching frame rate of the Mth of the first target layers and the optional refresh rates supported by the display screen.

7. The image display method according to claim 5, Characterized in that, The default refresh rate is 60Hz.

8. An image display method, characterized in that, it includes: Before each time the electronic device refreshes the content on the display screen, determine whether the electronic device is in a first device state, where the first device state includes that the electronic device receives a target interaction event or the display screen is playing a target dynamic effect, the target interaction event includes a click event and / or a first sliding event, and the first sliding event is a sliding event with a sliding speed greater than a first preset threshold; When the electronic device is in the first device state, refresh the content on the display screen at a first refresh rate, where the first refresh rate is the highest refresh rate of the display screen, or the first refresh rate is greater than or equal to a preset refresh rate threshold; When the electronic device is not in the first device state, determine whether the display screen includes a first target layer, where the first target layer is an active layer and the matching frame rate of the first target layer is greater than or equal to a first threshold, and the active layer includes a visible layer whose image content has been updated within a first historical duration; When the display screen includes at least one of the first target layers, respectively determine at least one expected refresh rate from at least one optional refresh rate based on the matching frame rate of each of the first target layers, and refresh the content on the display screen at the highest refresh rate among the at least one expected refresh rates; When the display screen does not include the first target layer, refresh the content on the display screen at a third refresh rate, where the third refresh rate is the default refresh rate of the display screen.

9. The image display method according to claim 8, characterized in that, Before determining whether the display screen includes a first target layer, the method further includes: Determine whether the display screen includes a second target layer, where the second target layer is an active layer playing a dynamic effect; The determination of whether the display screen includes a first target layer includes: When the display screen does not include the second target layer, determine whether the display screen includes the first target layer.

10. The image display method according to claim 8, characterized in that, The step of respectively determining at least one expected refresh rate from at least one optional refresh rate based on the matching frame rate of each of the first target layers includes: For each of the first target layers, when the matching frame rate is less than or equal to a second threshold, determine the optional refresh rate less than or equal to the second threshold as the candidate refresh rate, where the second threshold is greater than the first threshold; For each of the first target layers, when the matching frame rate is greater than the second threshold, determine the optional refresh rate greater than the second threshold as the candidate refresh rate; Score each of the candidate refresh rates based on the matching frame rate; Determine the candidate refresh rate with the highest score as the expected refresh rate of the first target layer.

11. The image display method according to claim 10, characterized in that, Scoring each of the candidate refresh rates based on the matching frame rate includes: When the candidate refresh rate is equal to the matching frame rate, determining that the candidate refresh rate is a first score value; When the candidate refresh rate is not equal to the matching frame rate, determining that the candidate refresh rate is a second score value, where the second score value is less than the first score value; Or, When the candidate refresh rate is an integer multiple of the matching frame rate, determining that the candidate refresh rate is the first score value; When the candidate refresh rate is not an integer multiple of the matching frame rate, determining that the candidate refresh rate is the second score value.

12. The image display method according to claim 8, characterized in that, further comprising: Before each time the electronic device refreshes the content on the display screen, obtaining a cache result, where the cache result includes the device state information of the electronic device, the layer state information of the active layer, and the historical refresh rate information used by the display screen before the previous refresh of the content on the display screen, where the screen refresh rate was a first historical refresh rate when the electronic device last refreshed the content on the display screen; Judging whether the number of current active layers is the same as the number of active layers before the previous refresh of the content on the display screen; When the number of active layers is different, determining the number of selectable refresh rates; When the number of active layers is the same, judging whether the layer state information of the current active layer is the same as the layer state information of the active layer before the previous refresh of the content on the display screen, and judging whether the device state information of the current electronic device is the same as the device state information of the electronic device before the previous refresh of the content on the display screen; When the layer state information of the active layer is the same and the device state information of the electronic device is the same, refreshing the content on the display screen at the first historical refresh rate.

13. The image display method according to claim 12, characterized in that, further comprising: When the layer state information of the active layer is different or the device state information of the electronic device is different, determining the number of selectable refresh rates; When there is only one selectable refresh rate, refreshing the content on the display screen at the selectable refresh rate.

14. The image display method according to claim 13, characterized in that, further comprising: When there are multiple selectable refresh rates, judging whether each layer on the display screen is in a static state and the electronic device has not received a touch event; When each layer on the display screen is in a static state and the electronic device has not received a touch event, refreshing the content on the display screen at the lowest refresh rate among the multiple selectable refresh rates. When at least one layer in the display screen is in a non - stationary state or the display screen receives a touch event, determine whether the active layer only includes a third target layer; wherein, the third target layer includes a wallpaper layer, a layer with a matching frame rate less than the first threshold, and a status bar layer; When the active layer only includes the third target layer, refresh the content in the display screen at the lowest refresh rate among the multiple optional refresh rates.

15. The image display method according to claim 14, wherein, The determination of whether the electronic device is in the first device state includes: when the active layer includes the third target layer and also includes other layers, determine whether the electronic device is in the first device state.

16. The image display method according to claim 15, wherein, further includes: When the electronic device receives a target interaction event or the display screen is playing a target dynamic effect, enter the first device state and start a first timer, and the first timer is set with a first timing duration; When the timing of the first timer exceeds the first timing duration, and / or when the electronic device receives a second sliding event, and / or when all layers in the display screen are in a stationary state, and / or when the target dynamic effect ends, the electronic device exits the first device state, and the second sliding event is a sliding event with a sliding speed less than or equal to the first preset threshold.

17. The image display method according to claim 12, wherein, Before obtaining the cache result, further includes: Before the electronic device refreshes the content in the display screen each time, for each active layer, determine whether the active layer contains matching frame rate information, and the matching frame rate information is used to indicate the matching frame rate; When the active layer does not contain the matching frame rate information, determine the matching frame rate based on the number of updates and the update time interval of the active layer within a second historical duration.

18. The image display method according to claim 17, wherein, The determination of the matching frame rate based on the number of updates and the update time interval of the active layer within a second historical duration includes: When the number of updates is greater than or equal to a number threshold or the time interval for each update is equal, the matching frame rate is the ratio of the number of updates to the second historical duration; When the number of updates is less than the number threshold and the time interval for each update is not equal, the matching frame rate is a first preset value.

19. The image display method according to claim 12, wherein, further includes: Save the device state information, the layer state information before the electronic device refreshes the content in the display screen each time, and the refresh rate adopted by the display screen when the electronic device refreshes the content in the display screen each time.

20. An electronic device, wherein, includes: A processor and a memory, the memory storing program instructions, which, when executed by the processor, cause the electronic device to execute the image display method according to any one of claims 1-19.

21. A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, which, when running on an electronic device, cause the electronic device to execute the image display method according to any one of claims 1-19.

22. A computer program product, characterized in that when the computer program product runs on an electronic device, it causes the electronic device to execute the image display method according to any one of claims 1-19.

Images