Interface display method and related device

Abstract

The invention provides an interface display method and a related device. The method comprises the following steps: when a display screen receives an Nth frame of image transmitted by a display driver, refreshing and displaying the Nth frame of image, wherein a frame rate corresponding to the Nth frame of image during drawing and rendering is a first frame rate; in the first duration of refreshing and displaying the Nth frame of image, under the condition that the display screen does not receive the (N + 1) th frame of image transmitted by the display driver, the display screen does not read the Nth frame of image from the cache and keeps displaying the Nth frame of image; the first duration is greater than the period duration corresponding to the first frame rate; after the display screen refreshes and displays the Nth frame of image for a first time length, when the display screen receives the (N + 1) th frame of image transmitted by the display driver, the display screen refreshes and displays the (N + 1) th frame of image, the frame rate corresponding to the (N + 1) th frame of image during drawing and rendering is a second frame rate, and the second frame rate is smaller than the first frame rate. The refresh rate is adaptively adjusted, the image refresh frequency is reduced, and power consumption is reduced.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an interface display method and related devices. Background Art

[0002] Currently, terminal devices can support a variety of screen refresh rates, such as 60 times per second (Hz), 90 Hz, 120 Hz, or 144 Hz. The screen refresh rate may be consistent with the application frame rate, or it may be inconsistent with the application frame rate.

[0003] How to reduce the power consumption of terminal devices based on the relationship between screen refresh rate and application frame rate is a problem that needs to be solved. Summary of the invention

[0004] The embodiment of the present application provides an interface display method and related devices, which are applied to the field of terminal technology. The display driver chip of the display screen can adaptively adjust the refresh rate. When the display driver does not send an image for display, the image is not read from the cache for image refresh until the preset time is reached or an image is sent for display. In this way, image refresh based on the image sending time can reduce the number of image refreshes when there is no new synthetic image sent for display, thereby reducing the power consumption of the display screen.

[0005] In the first aspect, the embodiment of the present application proposes an interface display method. The method includes: when the display screen receives the image transmitted by the display driver, the display screen refreshes and displays the image transmitted by the display driver; wherein, when the display screen receives the Nth frame image transmitted by the display driver, the display screen refreshes and displays the Nth frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is a first frame rate; within the first duration of refreshing and displaying the Nth frame image, if the display screen does not receive the N+1th frame image transmitted by the display driver, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate; after the first duration after the display screen refreshes and displays the Nth frame image, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image, and the frame rate corresponding to the drawing and rendering of the N+1th frame image is a second frame rate, and the second frame rate is less than the first frame rate.

[0006] In this way, when the application frame rate becomes smaller, the refresh rate of the display screen is adjusted to be lower, the image refresh frequency of the display screen is reduced, and power consumption is reduced. The terminal device can adaptively adjust the refresh rate. In some scenarios, the refresh rate is higher to meet the fluency requirements; in some scenarios, the refresh rate is lower to reduce power consumption. In addition, the refresh rate of the display screen is adjusted based on the image sent to the display, and there is no need to identify the application frame rate. The above method can be used in low frame rate scenarios such as video and camera, covering more scenarios, with a wide range of applications and high practicality.

[0007] Optionally, before the display screen refreshes the display when receiving the image transmitted by the display driver, the method also includes: the terminal device obtains the brightness of the display screen, and / or the intensity of ambient light; the display screen refreshes the display when receiving the image transmitted by the display driver, including: when the brightness of the display screen is greater than a first value, and / or the ambient light intensity is greater than a second value, the display screen refreshes the display when receiving the image transmitted by the display driver.

[0008] In this way, the terminal device can achieve adaptive refresh in scenarios where the display brightness and / or ambient light intensity meet the conditions, thereby reducing power consumption; in other scenarios, adaptive refresh is not used to reduce problems such as screen flickering caused by refresh rate switching.

[0009] Optionally, the method also includes: the terminal device determines a first refresh rate based on the brightness of the display screen and / or the ambient light intensity; when the display screen does not receive the N+1th frame image transmitted by the display driver within a third time period after refreshing and displaying the Nth frame image, the display screen reads the Nth frame image from the cache and refreshes and displays the Nth frame image within a third time period after refreshing and displaying the Nth frame image, wherein the third time period is the cycle time period corresponding to the first refresh rate.

[0010] In this way, the minimum refresh rate is set based on the display brightness and / or the ambient light intensity, and the energy consumption is reduced as much as possible while reducing the screen flicker of the terminal device. The image refresh display flicker caused by the excessive refresh time during adaptive refresh can be reduced.

[0011] Optionally, the terminal device stores a first corresponding relationship; the first corresponding relationship is the relationship between the display screen brightness and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness and the first corresponding relationship; or, the terminal device stores a second corresponding relationship; the second corresponding relationship is the relationship between the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the ambient light intensity and the second corresponding relationship; or, the terminal device stores a third corresponding relationship; the third corresponding relationship is the relationship between the display screen brightness, the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness, the ambient light intensity and the third corresponding relationship.

[0012] In this way, it is convenient for the terminal device to confirm the minimum refresh rate, which is simple and easy to implement.

[0013] Optionally, the method also includes: when the brightness of the display screen is less than or equal to the first value, and the ambient light intensity is less than or equal to the second value, the display screen refreshes the display at a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device; wherein, when the display screen receives the Mth frame image transmitted by the display driver, it refreshes and displays the Mth frame image, and the frame rate corresponding to the drawing and rendering of the Mth frame image is a third frame rate; within a second time period after refreshing and displaying the Mth frame image, if the display screen does not receive the M+1th frame image transmitted by the display driver, the display screen reads the Mth frame image from the cache and refreshes and displays the Mth frame image within a second time period after refreshing and displaying the first application interface, and the second time period is equal to the cycle time corresponding to the third frame rate; after receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is a fourth frame rate, and the fourth frame rate is less than the third frame rate.

[0014] In this way, when the application frame rate becomes smaller, the terminal device can adjust the refresh rate of the display screen to reduce the image refresh frequency of the display screen and reduce power consumption.

[0015] Optionally, the terminal device pre-stores an application list; the application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

[0016] In this way, the terminal device can confirm whether to perform adaptive refresh based on the application.

[0017] Optionally, the Nth frame image is an image including barrage, and the N+1th frame image is an image not including barrage.

[0018] In this way, when there is a barrage, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there is no barrage, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0019] Optionally, the Nth frame image is an image corresponding to a first scene; the N+1th frame image is an image corresponding to a second scene, the first scene is a battle scene of a game application, and the second scene is a non-battle scene of the game application.

[0020] In this way, in scenes with high requirements for smoothness, such as game battle scenes, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; in scenes with lower requirements for smoothness, such as game lobby scenes, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0021] Optionally, the Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

[0022] In this way, when there are motion effects, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there are no motion effects, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0023] In a second aspect, an embodiment of the present application provides a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc.

[0024] The terminal device includes:

[0025] The terminal device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of the first aspect.

[0026] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0027] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute a method such as the fourth aspect.

[0029] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram showing the flow chart for one possible design;

[0031] Figure 2 A schematic diagram of a display process provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of a software framework of a terminal device provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the flow of internal module interaction corresponding to an interface display method provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a display process provided in an embodiment of the present application;

[0035] Figure 6 A flowchart of an interface display method provided in an embodiment of the present application;

[0036] Fig. 7A A schematic diagram of an application scenario provided for an embodiment of the present application;

[0037] Figure 7B A schematic diagram of a display process provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0039] Fig. 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] To facilitate understanding of the embodiments of the present application, some terms involved in the present application are first briefly explained.

[0041] 1. Frame: refers to the smallest unit of a single picture in the interface display. A frame can be understood as a still picture. Displaying multiple connected frames quickly and continuously can create the illusion of object movement. Frame rate refers to the number of frames that refresh the picture in 1 second, and can also be understood as the number of times the graphics processor in the terminal device refreshes the picture per second. A high frame rate can produce smoother and more realistic animations. The more frames per second, the smoother the displayed action will be.

[0042] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0043] 2. Drawing: refers to the picture drawing of the display interface. The display interface can be composed of one or more views, each view can be drawn by the visual control of the view system, each view is composed of subviews, and a subview corresponds to a small widget in the view, for example, one of the subviews corresponds to a symbol in the picture view.

[0044] 3. Rendering: It is to color the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0045] 4. Synthesis: It is the process of synthesizing multiple or more rendered views into a display interface.

[0046] 5. Other terms

[0047] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0048] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0049] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0050] 6. Terminal equipment

[0051] The terminal device of the embodiment of the present application may also be any form of electronic device. For example, the electronic device may include a handheld device, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public land mobile The embodiments of the present application do not limit this.

[0052] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0053] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0054] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0055] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0056] Currently, terminal devices can support a variety of screen refresh rates, such as 60 times per second (Hz), 90 Hz, 120 Hz, or 144 Hz. The screen refresh rate may be consistent with the application frame rate, or it may be inconsistent with the application frame rate.

[0057] In scenarios where the frame rate of applications such as video and camera is low, there may be content such as bullet screen and animation that needs to be displayed. In order to make the content such as bullet screen and animation display smoothly, the screen refresh rate in scenarios where the frame rate of applications such as video and camera is low is usually higher than the application frame rate.

[0058] In a possible design, in a video playback scenario, if there is a bullet screen, the terminal device synthesizes and displays the image at a frequency of 60 frames per second to improve the smoothness of the bullet screen display. When the bullet screen disappears, the terminal device synthesizes the image at a frequency of 30 frames per second, and the terminal device continues to refresh the displayed image at a frequency of 60 frames per second, which consumes a lot of power. In addition, when the terminal device pauses the video playback, the display screen of the terminal device will continue to refresh the displayed image at a frequency of 60 frames per second, which consumes a lot of power.

[0059] For example, Figure 1 The following is a possible display flow diagram of the design. Take the video as an example, which is played at 30 frames per second and the bullet comments are displayed at 60 frames per second. Figure 1 In the video, the video corresponds to frame 1, frame 2, and frame 3. The bullet comments correspond to bullet comments 1, bullet comments 2, and bullet comments 3. Since the bullet comments are displayed at 60 frames per second, the display screen is refreshed at 60 frames per second.

[0060] At 0ms, the image synthesis service (surface flinger) in the terminal device synthesizes frame 1 and barrage 1.

[0061] At 16.6ms, the image synthesis service synthesizes frame 1 and bullet comment 2, and the display driver transmits the synthesized frame 1 and bullet comment 1 to the display screen for display. The display screen starts to display frame 1 and bullet comment 1.

[0062] At 33.2ms, the image synthesis service synthesizes frame 2 and bullet comment 3, and the display driver transmits the synthesized frame 1 and bullet comment 2 to the display screen for display. The display screen starts to display frame 1 and bullet comment 2.

[0063] At 49.8ms, the bullet screen display is complete, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 2 and bullet screen 3 to the display screen for display. The display screen starts to display frame 2 and bullet screen 3.

[0064] At 64.4ms, the image synthesis service synthesizes frame 3. Since the image synthesis service did not perform synthesis at 49.8ms, the display driver did not transmit the synthesized image to the display. The display reads display frame 2 and bullet screen 3 from the cache and refreshes display frame 2 and bullet screen 3.

[0065] At 83.0ms, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 3 to the display screen. The display screen refreshes and displays frame 3.

[0066] At 99.6ms, the image synthesis service synthesizes frame 4. Since the image synthesis service did not perform the synthesis operation at 83.0ms, the display driver did not transmit the synthesized image to the display. The display reads frame 3 from the cache and refreshes the display frame 3.

[0067] At 116.2ms, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 4 to the display screen. The display screen refreshes and displays frame 4.

[0068] from Figure 1 It can be seen that at 64.4ms and 99.6ms, when there is no new image to be displayed, the display screen of the terminal device will still read the image from the cache of the display screen and refresh the displayed image. This makes the power consumption of the terminal device higher.

[0069] In view of this, the embodiment of the present application provides a refresh rate switching method and related device. The display driver chip of the display screen can adaptively adjust the refresh rate. When the display driver does not send an image for display, the image is not read from the cache for image refresh until the preset time is reached or an image is sent for display. In this way, image refresh based on the image display time can reduce the number of image refreshes when there is no new synthetic image sent for display, thereby reducing the power consumption of the display screen.

[0070] Combine the following Figure 2 The display process of adaptive refresh is explained.

[0071] For example, Figure 2 A display flow diagram provided in an embodiment of the present application. Taking the video as played at 30 frames per second and the bullet screen as displayed at 60 frames per second as an example, Figure 2 In the video, the video corresponds to frame 1, frame 2, and frame 3. The bullet comments correspond to bullet comments 1, bullet comments 2, and bullet comments 3. Since the bullet comments are displayed at 60 frames per second, the display screen is refreshed at 60 frames per second.

[0072] Figure 2 In the video, the video corresponds to frame 1, frame 2, and frame 3. The bullet comments correspond to bullet comments 1, bullet comments 2, and bullet comments 3. Since the bullet comments are displayed at 60 frames per second, the display screen is refreshed at 60 frames per second.

[0073] At 0ms, the image synthesis service (surface flinger) in the terminal device synthesizes frame 1 and barrage 1.

[0074] At 16.6ms, the image synthesis service synthesizes frame 1 and bullet comment 2, and the display driver transmits the synthesized frame 1 and bullet comment 1 to the display screen for display. The display screen starts to display frame 1 and bullet comment 1.

[0075] At 33.2ms, the image synthesis service synthesizes frame 2 and bullet comment 3, and the display driver transmits the synthesized frame 1 and bullet comment 2 to the display screen for display. The display screen starts to display frame 1 and bullet comment 2.

[0076] At 49.8ms, the bullet screen display is complete, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 2 and bullet screen 3 to the display screen for display. The display screen starts to display frame 2 and bullet screen 3.

[0077] At 64.4ms, the image synthesis service synthesizes frame 3. Since the image synthesis service did not perform the synthesis operation at 49.8ms, the display driver does not transmit the synthesized image to the display. The display does not read frame 2 and bullet screen 3 from the cache, does not refresh the image, and keeps displaying frame 2 and bullet screen 3.

[0078] At 83.0ms, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 3 to the display screen. The display screen refreshes and displays frame 3.

[0079] At 99.6ms, the image synthesis service synthesizes frame 4. Since the image synthesis service did not perform the synthesis operation at 83.0ms, the display driver does not transmit the synthesized image to the display. The display does not read frame 3 from the cache, does not refresh the image, and keeps displaying frame 3.

[0080] At 116.2ms, the image synthesis service does not perform synthesis, and the display driver transmits the synthesized frame 4 to the display screen. The display screen refreshes and displays frame 4.

[0081] from Figure 2 It can be seen that at 64.4ms and 99.6ms, when there is no image sent to the terminal device display screen, the display screen does not read the image from the cache and does not refresh the image. Figure 1 The display process shown, Figure 2 In the process shown, the image refresh times of the display screen are relatively small, which can reduce the power consumption of the terminal device.

[0082] Combine the following Figure 3-7B The method provided in the embodiments of the present application is described.

[0083] For ease of understanding, the software framework of the terminal device is described below. The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.

[0084] Taking the layered architecture of the Android system as an example, the software structure of the terminal device is exemplified. Figure 3 A schematic diagram of the structure of a software framework of a terminal device provided in an embodiment of the present application. Figure 3 As shown in FIG. 1 , the layered architecture divides the software system of the terminal device into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, namely, application layer (applications), application framework layer (application framework), hardware abstract layer (hardware abstract layer, HAL), kernel layer (kernel) and hardware layer.

[0085] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. Figure 3 As shown, the application package may include applications such as camera, calendar, map, phone, music, settings, notes, video, etc.

[0086] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, an image synthesis service, a frame rate control service, a content provider, a phone manager, a resource manager, a notification manager, and the like.

[0087] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0088] The image synthesis service is used to synthesize images and transmit the synthesized images to the display driver to display the images on the display screen. Exemplarily, the image synthesis service can synthesize images rendered by one or more applications.

[0089] The frame rate control service is used to adjust the refresh rate of the display screen through the display driver. In some embodiments, the frame rate control service includes: a brightness ambient light processing module. The brightness ambient light processing module is used to control the adaptive switch in the display driver to turn on or off based on the brightness of the display screen and the ambient light intensity of the terminal device to adjust the refresh strategy of the display screen. When the brightness of the display screen is greater than the first value and the ambient light intensity is greater than the second value, the brightness ambient light processing module controls the adaptive switch in the display driver to turn on to achieve adaptive refresh of the display screen; when the brightness of the display screen is less than or equal to the first value, or the ambient light intensity is less than or equal to the second value, the brightness ambient light processing module controls the adaptive switch in the display driver to turn off, and the display screen refreshes at a certain frequency.

[0090] The sensor service is used to obtain the display brightness of the terminal device and the ambient light intensity of the terminal device.

[0091] Content providers are used to store and retrieve data and make it accessible to applications. Data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls that display text, controls that display images, etc.

[0092] The phone manager is used to provide communication functions for terminal devices, such as the management of call status (including answering, hanging up, etc.).

[0093] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc.

[0094] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0095] The hardware abstraction layer can contain multiple library modules, such as hardware synthesizers. The Android system can load the corresponding library modules for the device hardware, thereby enabling the application framework layer to access the device hardware. The device hardware can include, for example, the display screen in the terminal device.

[0096] The kernel layer is a layer between hardware and software. The kernel layer is used to drive the hardware to make the hardware work. The kernel layer at least includes display driver, camera driver, audio driver, sensor driver, etc., which is not limited in the embodiments of the present application.

[0097] In some embodiments, the display driver includes an adaptive switch, and the adaptive switch is used to control the refresh strategy of the display screen. When the adaptive switch is turned on, the display screen does not refresh when the display driver does not send an image, until a preset time is reached or the display driver sends an image. When the adaptive switch is turned off, the display screen refreshes at a certain frequency, and refreshes even when the display driver does not send an image.

[0098] Combine the following Figure 3 The display process of the terminal device is described.

[0099] The application calls the image rendering library in the system library to render the image through the view system of the application framework layer, and transmits the rendered image to the cache queue of the image synthesis service. The image synthesis service synthesizes the rendered images of one or more applications and transmits the synthesized image to the display driver. The display driver transmits the synthesized image to the display screen for display.

[0100] For example, Figure 4 A schematic diagram of the flow of internal module interaction corresponding to an interface display method provided in an embodiment of the present application. The terminal device includes: a frame rate control service, a display driver, an application, an image synthesis service, and a display screen. Figure 4 As shown, the interaction process includes:

[0101] S401. After booting up, the frame rate control service registers a callback with the sensor service to obtain the display brightness of the terminal device and / or the ambient light intensity of the terminal device.

[0102] In some embodiments, after powering on, the frame rate control service registers a callback with the sensor service to facilitate subsequent updating of the display brightness and / or the ambient light intensity, or obtaining display brightness change information and / or ambient light intensity change information when the display brightness of the terminal device changes or the ambient light intensity of the terminal device changes.

[0103] In other embodiments, the frame rate control service may also obtain the display brightness and / or ambient light intensity of the terminal device from the sensor service at a preset time interval. The embodiment of the present application does not limit the manner in which the frame rate control service obtains the display brightness and ambient light intensity.

[0104] S402: The sensor service transmits display screen brightness information and / or ambient light intensity information to the frame rate control service.

[0105] The display screen brightness information is used to indicate the brightness of the display screen or to indicate the brightness change of the display screen. The ambient light intensity information is used to indicate the ambient light intensity or to indicate the ambient light intensity change.

[0106] The frame rate control service determines the refresh strategy of the terminal device's display based on the display brightness, ambient light intensity, and preset conditions.

[0107] When the display brightness is greater than the first value and / or the ambient light intensity is greater than the second value, the terminal device executes S403-S405. When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the terminal device executes S406-S405.

[0108] S403: When the display screen brightness is greater than a first value and / or the ambient light intensity is greater than a second value, the frame rate control service transmits a message to the display driver for instructing the adaptive switch to be turned on.

[0109] The first value may be 50 nit, 60 nit, or any other value. The second value may be 50 lux, 60 lux, or any other value. The embodiment of the present application does not limit the specific values ​​of the first value and the second value.

[0110] It should be noted that the human eye perceives the terminal device differently under different display brightness and ambient light intensity. If the light is dim or the display brightness is low, the human eye may perceive flickering when the terminal device switches the refresh rate, which reduces the user experience.

[0111] It is understandable that the terminal device can confirm whether the adaptive switch is turned on or off based only on the display brightness; the terminal device can also confirm whether the adaptive switch is turned on or off based only on the ambient light intensity; the terminal device can also confirm whether the adaptive switch is turned on or off based on the display brightness and the ambient light intensity. This embodiment of the application is not limited to this.

[0112] In some embodiments, the frame rate control service transmits a message to the display driver through an interface.

[0113] S404: After receiving the message for instructing the adaptive switch to be turned on, the display driver transmits a message for instructing adaptive refresh to the display screen.

[0114] S405 . When receiving the synthesized image transmitted by the display driver, the display screen refreshes and displays the synthesized image.

[0115] In the embodiment of the present application, the display screen includes a display driver IC (DDIC), and the display driver chip can control the refresh of the display screen of the terminal device and the refresh interval.

[0116] S406: When the brightness of the display screen is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the frame rate control service transmits a message indicating a preset refresh rate to the display driver.

[0117] S407: After receiving the message indicating the preset refresh rate, the display driver transmits the message indicating the preset refresh rate to the display screen.

[0118] S408: The display screen refreshes and displays the synthesized image at a preset refresh rate.

[0119] The preset refresh rate can be 60 Hz, 120 Hz, or any other value, which is not limited in the embodiment of the present application.

[0120] In a possible implementation, the preset refresh rate is related to the foreground application of the terminal device. When the foreground application is a video application, the preset refresh rate may be 60 Hz, and when the foreground application is a game application, the preset refresh rate may be 120 Hz.

[0121] The preset refresh rate may also be related to the operating load of the terminal device, etc. The embodiment of the present application does not limit the method for determining the preset refresh rate.

[0122] In this way, the terminal device can control adaptive refresh when the display screen brightness is high and / or the ambient light intensity is high, reduce the refresh frequency of the display screen, and reduce power consumption. The terminal device can control the terminal device to refresh at a preset refresh rate when the display screen brightness is low and the ambient light intensity is low, reducing problems such as screen flickering caused by refresh rate switching.

[0123] Combine the following Figure 5 For the above Figure 4 The two refresh strategies are provided for illustration. Take the case where the image synthesis service (surface flinger) synthesizes frame 1, frame 2, frame 3, frame 4, and frame 5 at 0ms, 32.2ms, 49.8ms, 64.4ms, and 99.6ms, respectively.

[0124] When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the display process of the terminal device can be as follows: Figure 5 Specifically, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display driver transmits synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5 to the display screen, respectively. Adaptively, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display screen starts to display synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5, respectively.

[0125] The display screen did not receive the synthesized frame transmitted by the display driver at 32.2ms, read the synthesized frame 1 from the cache, refreshed, and displayed the synthesized frame 1. The display screen did not receive the synthesized frame transmitted by the display driver at 99.6ms, read the synthesized frame 4 from the cache, and refreshed to continue displaying the synthesized frame 4.

[0126] In this way, the refresh rate of the display is fixed, which can reduce problems such as screen flickering caused by refresh rate switching and improve user experience.

[0127] When the display brightness is greater than the first value or the ambient light intensity is greater than the second value, the display process of the terminal device can be as follows: Figure 5 Specifically, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display driver transmits synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5 to the display screen, respectively. Adaptively, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display screen starts to display synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5, respectively.

[0128] The display screen does not receive the synthesized frame transmitted by the display driver at 32.2ms, does not read the synthesized frame from the cache, does not refresh the image, and keeps displaying synthesized frame 1. At 99.6ms, the display screen does not receive the synthesized frame transmitted by the display driver, does not read the synthesized frame from the cache, does not refresh the image, and keeps displaying synthesized frame 4.

[0129] Compared to Figure 5 The process shown in a in Figure 5 The process shown in b can reduce two image refreshes and reduce the power consumption of the terminal device.

[0130] It should be noted that after the display screen refreshes the image, the voltage of the light-emitting device in the display screen decays over time, and the brightness of the image decreases accordingly. When the terminal device refreshes the image again, the voltage of the light-emitting device in the display screen increases. When the image is not refreshed for a long time, the voltage difference of the light-emitting device in the display screen before and after the image is refreshed is large, and the brightness difference is large, which makes the human eye feel abnormalities such as screen flickering, and thus reduces the user experience of the terminal device.

[0131] On the basis of the above embodiment, the message for indicating that the adaptive switch is turned on in S403 includes: the minimum refresh rate. The message for indicating adaptive refresh in S407 includes: the minimum refresh rate. Adaptively, S405 also includes: when the display screen does not receive the synthesized image within the first preset time after refreshing the image, the frame image is read from the cache, and the image is refreshed to display the frame image. The first preset time is related to the minimum refresh rate. The first preset time can be the reciprocal of the minimum refresh rate.

[0132] The minimum refresh rate can be 10 Hz, 30 Hz, or any other value, which is not limited in the embodiments of the present application.

[0133] In this way, limiting the minimum refresh rate of adaptive refresh can reduce abnormal phenomena such as screen flickering on terminal devices and improve user experience.

[0134] In the embodiment of the present application, the refresh rate of the display screen can be 1 Hz, 10 Hz, 30 Hz, 40 Hz, 60 Hz, 120 Hz, etc. The embodiment of the present application does not limit the specific value of the refresh rate of the display screen.

[0135] In some embodiments, the frame rate control service may also determine a minimum refresh rate based on display brightness and / or ambient light intensity.

[0136] In a possible implementation manner 1, the frame rate control service stores a first corresponding relationship, which is a relationship between display screen brightness and a minimum refresh rate.

[0137] For example, the display brightness is 50 nit and the minimum refresh rate is 10 Hz; the display brightness is 60 nit and the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the first corresponding relationship.

[0138] In this way, the minimum refresh rate is set based on the brightness of the display screen to reduce energy consumption as much as possible while reducing screen flickering on the terminal device.

[0139] In possible implementation manner 2, the frame rate control service stores a second corresponding relationship, which is a relationship between ambient light intensity and the minimum refresh rate.

[0140] For example, when the ambient light intensity is 50 lux, the minimum refresh rate is 10 Hz; when the ambient light intensity is 60 lux, the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the second corresponding relationship.

[0141] In this way, the minimum refresh rate is set based on the ambient light intensity, which can reduce energy consumption as much as possible while reducing the screen flicker of the terminal device.

[0142] In possible implementation manner three, the frame rate control service stores a third corresponding relationship, which is a relationship between display brightness, ambient light intensity, and the minimum refresh rate.

[0143] For example, the display brightness is 50 nit, the ambient light intensity is 50 lux, and the minimum refresh rate is 10 Hz; the display brightness is 60 nit, the ambient light intensity is 60 lux, and the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the third corresponding relationship.

[0144] In this way, the minimum refresh rate is set based on the display brightness and ambient light intensity, which can reduce energy consumption as much as possible while reducing screen flickering of terminal devices.

[0145] Based on the above embodiment, the frame rate control service can also determine the minimum refresh rate based on the highest frame rate corresponding to the foreground running application. Under the same conditions, the higher the highest frame rate corresponding to the foreground running application, the higher the minimum refresh rate.

[0146] In this way, the refresh time difference and refresh rate difference during adaptive refresh can be reduced, and the flicker during frame rate switching can be further reduced.

[0147] Based on the above embodiment, the activation of the adaptive refresh is also related to the foreground running application. Alternatively, it can be understood that the terminal device can control the display screen to adaptively refresh based on the specific application.

[0148] In some embodiments, the frame rate control service stores an application list. When the foreground running application is an application in the application list, the frame rate control service confirms whether the adaptive switch is turned on or off based on the display brightness and / or the ambient light intensity. When the foreground running application is not an application in the application list, the frame rate control service transmits a message to the display driver for indicating the preset refresh rate.

[0149] For example, Figure 6 The following is a flow chart of an interface display method provided in an embodiment of the present application. Figure 6 As shown, the method includes:

[0150] S601. When receiving an image from a display driver, the display screen of the terminal device refreshes and displays the image transmitted by the display driver.

[0151] S602: When receiving the Nth frame image from the display driver, the display screen refreshes and displays the Nth frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is the first frame rate.

[0152] S603: within the first duration of refreshing and displaying the Nth frame image, if the display screen does not receive the N+1th frame image transmitted by the display driver, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate;

[0153] S604. After the display screen refreshes and displays the Nth frame image for a first period of time, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image, and the corresponding frame rate when the N+1th frame image is drawn and rendered is the second frame rate, and the second frame rate is less than the first frame rate.

[0154] It can be understood that the first frame rate can be 60 frames / second, 120 frames / second, or any other value, and the second frame rate can be 30 frames / second, 60 frames / second, or any other value, and the implementation of this application is not limited to this.

[0155] In this way, when the application frame rate becomes smaller, the terminal device can adjust the refresh rate of the display screen to reduce the image refresh frequency of the display screen and reduce power consumption.

[0156] In addition, the refresh rate of the display screen is adjusted based on the image being displayed, without the need to identify the application frame rate. The above method can be used in low frame rate scenes such as video and camera, covering a large number of scenes, with a wide range of applications and high practicality.

[0157] Optionally, the terminal device pre-stores an application list; the application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

[0158] In this way, the terminal device can confirm whether to perform adaptive refresh based on the application.

[0159] Optionally, the Nth frame image is an image including barrage, and the N+1th frame image is an image not including barrage.

[0160] In this way, when there is a barrage, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there is no barrage, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0161] Optionally, the Nth frame image is an image corresponding to a first scene; the N+1th frame image is an image corresponding to a second scene, the first scene is a battle scene of a game application, and the second scene is a non-battle scene of the game application.

[0162] In this way, in scenes with high requirements for smoothness, such as game battle scenes, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; in scenes with lower requirements for smoothness, such as game lobby scenes, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0163] Optionally, the Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

[0164] In this way, when there are motion effects, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there are no motion effects, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0165] Based on the above embodiment, the terminal device can also confirm that the display screen is adaptively refreshed or refreshed at a certain refresh rate based on the display screen brightness and / or the ambient light intensity, and confirm to execute S601-S604 or execute the following S605-S608.

[0166] In this way, the terminal device can achieve adaptive refresh in some scenarios to reduce power consumption; in other scenarios, it can refresh at a certain refresh rate to reduce problems such as screen flickering caused by refresh rate switching.

[0167] In possible implementation mode 1, when the display screen brightness is greater than the first value, the terminal device executes S601 to S604. When the display screen brightness is less than or equal to the first value, the terminal device executes S605 to S608.

[0168] S605, the display screen refreshes the display according to a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device;

[0169] S606: When the display screen receives the Mth frame image transmitted by the display driver, it refreshes and displays the Mth frame image, and the frame rate corresponding to the rendering of the Mth frame image is the third frame rate;

[0170] S607: If the display screen does not receive the M+1th frame image transmitted by the display driver within the second time duration after refreshing the display of the Mth frame image, the display screen reads the Mth frame image from the cache and refreshes the display of the Mth frame image during the second time duration after refreshing the display of the first application interface, and the second time duration is equal to the cycle duration corresponding to the third frame rate;

[0171] S608. After receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image, and the corresponding frame rate when the Nth frame image is drawn and rendered is the fourth frame rate, and the fourth frame rate is less than the third frame rate. It can be understood that the third frame rate can be 60 frames / second, or 120 frames / second or any other value, and the fourth frame rate can be 30 frames / second, or 60 frames / second or any other value, and the implementation of this application does not limit this.

[0172] In this way, the terminal device can control adaptive refresh when the display screen brightness is high, reduce the refresh frequency of the display screen, and reduce power consumption. When the display screen brightness is low, the terminal device can control the terminal device to refresh at a preset refresh rate to reduce problems such as screen flickering caused by switching the refresh rate.

[0173] In possible implementation mode 2, when the ambient light intensity is greater than the second value, the terminal device executes S601 to S604. When the ambient light intensity is less than or equal to the second value, the terminal device executes S605 to S608.

[0174] In this way, the terminal device can control adaptive refresh when the ambient light intensity is high, reduce the refresh frequency of the display screen, and reduce power consumption. When the ambient light intensity is low, the terminal device can control the terminal device to refresh at a preset refresh rate to reduce problems such as screen flickering caused by switching the refresh rate.

[0175] In possible implementation mode 3, when the display brightness is greater than the first value or the ambient light intensity is greater than the second value, the terminal device executes S601-S604. When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the terminal device executes S605-S608.

[0176] In this way, the terminal device can control adaptive refresh when the display brightness is high or the ambient light intensity is high, reduce the refresh frequency of the display, and reduce power consumption. When the display brightness is low and the ambient light intensity is low, the terminal device can control the terminal device to refresh at a preset refresh rate to reduce problems such as screen flickering caused by refresh rate switching.

[0177] In possible implementation mode 4, when the display brightness is greater than the first value and the ambient light intensity is greater than the second value, the terminal device executes S601-S604. When the display brightness is less than or equal to the first value, or the ambient light intensity is less than or equal to the second value, the terminal device executes S605-S608.

[0178] In this way, the terminal device can control adaptive refresh when the display brightness is high and the ambient light intensity is high, reduce the refresh frequency of the display, and reduce power consumption. The terminal device can control the terminal device to refresh at a preset refresh rate when the display brightness is low or the ambient light intensity is low, reducing problems such as screen flickering caused by refresh rate switching.

[0179] The terminal device can confirm the adaptive refresh or preset frequency refresh of the display screen through any one of the possible implementation methods 1 to 4 above. This is not limited here.

[0180] The adaptive refresh after satisfying preset conditions provided by the embodiment of the present application is described below in conjunction with specific application scenarios.

[0181] For example, Fig. 7A Schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario is a video playback scenario. Fig. 7A As shown in a in FIG. 1 , in this application scenario, the terminal device displays a first video 701. During the playback of the first video 701, the terminal device may display a bullet screen 701 (such as Fig. 7A (as shown in b in the figure).

[0182] It is understandable that when the video is played, the video application draws and renders the video frame at the first frame rate, and the terminal device synthesizes the video frame at the first frame rate. When there is a bullet comment to be displayed, the terminal device synthesizes the video frame and the bullet comment at the second frame rate.

[0183] For example, Figure 7B A display process diagram provided in an embodiment of the present application. The video application draws and renders video frames at 30 frames per second. The terminal device draws and renders bullet screens at 60 frames per second. Figure 7B As shown, the video frames correspond to frame 1, frame 2, frame 3, and frame 4; the barrage corresponds to barrage 1, barrage 2, and barrage 3.

[0184] like Figure 7B As shown, at 0ms, the video application draws rendering frame 1.

[0185] At 16.6ms, the video application does not perform drawing and rendering. Since the drawing and rendering of frame 1 is completed and the drawing and rendering of bullet comment 1 is completed, the image synthesis service synthesizes frame 1 and bullet comment 1.

[0186] At 33.2ms, the video application draws and renders frame 2. Since frame 2 has not been drawn and rendered, bullet comment 2 has been drawn and rendered, and the image synthesis service synthesizes frame 1 and bullet comment 2. Since the synthesis of frame 1 and bullet comment 1 has been completed, the display driver transmits the synthesized frame 1 and bullet comment 1 to the display screen for display. The display screen starts to display frame 1 and bullet comment 1.

[0187] At 49.8ms, the video application does not perform drawing and rendering. Since the drawing and rendering of frame 2 is completed and the drawing and rendering of bullet screen 3 is completed, the image synthesis service synthesizes frame 2 and bullet screen 3. Since the synthesis of frame 1 and bullet screen 2 is completed, the display driver transmits the synthesized frame 1 and bullet screen 2 to the display screen for display. The display screen starts to display frame 1 and bullet screen 2.

[0188] At 64.4ms, the video application draws and renders frame 3. Since frame 3 has not been drawn and rendered and no bullet screen has been drawn and rendered, the synthesis service has no frames to be synthesized and does not synthesize. Since the synthesis of frame 2 and bullet screen 3 is completed, the display driver transmits the synthesized frame 2 and bullet screen 3 to the display screen for display. The display screen starts to display frame 2 and bullet screen 3.

[0189] Since the 49.8ms image synthesis service did not perform the synthesis operation, the display driver did not transmit the synthesized image to the display screen. The display screen did not refresh and continued to display frame 2 and bullet screen 3.

[0190] At 83.0ms, the video application does not perform drawing and rendering. Since the drawing and rendering of frame 3 is completed, the image synthesis service synthesizes frame 3. Since the image synthesis service does not perform synthesis operation at 64.4ms, the display driver does not transmit any image to the display screen, and the display screen does not refresh and continues to display frame 2 and bullet screen 3.

[0191] At 99.6ms, the video application draws and renders frame 4. Since frame 4 has not been drawn and rendered and no bullet screen drawing and rendering has been completed, the synthesis service has no frames to be synthesized and does not synthesize. Since the synthesis of frame 3 is completed, the display driver transmits the synthesized frame 3 to the display screen for display. The display screen refreshes and starts to display frame 3.

[0192] At 116.2ms, the video application does not perform drawing and rendering. Since the drawing and rendering of frame 4 is completed, the image synthesis service synthesizes frame 4. Since the image synthesis service does not perform synthesis operation at 99.6ms, the display driver does not transmit any image to the display screen, and the display screen does not refresh and continues to display frame 3.

[0193] from Figure 7B It can be seen that the terminal device adaptively adjusts the refresh rate based on whether the barrage is displayed or not. Specifically, the display screen refreshes the barrage at 32.2ms, 49.8ms, and 64.4ms. The time interval for the barrage display is 16.6ms. When the barrage refresh display ends, the display screen refreshes the display frame 3 at 99.6ms, and the time interval for the video frame refresh display is 32.2ms. Or it can be understood that the corresponding refresh rate when the barrage is refreshed is 30Hz, and the corresponding refresh rate of the video frame when there is no barrage refresh display is 60Hz.

[0194] In this way, the terminal device can adaptively adjust the refresh rate based on whether the bullet screen is displayed or not, which can reduce the power consumption of the terminal device. In addition, there is no need to adjust the frame rate of the image synthesized by the image synthesis service, the frame rate of the application when drawing and rendering the image, etc. It is easy to implement and has a fast refresh rate response speed.

[0195] For example, Figure 8 Schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario is a live broadcast scenario. Figure 8 As shown in a in FIG. 1 , in this application scenario, the terminal device displays a live video 801 and comments 802 (equivalent to displaying bullet comments). When the user slides his finger to the right on the display screen, the terminal device receives an operation for instructing to clear the screen (clearing the screen is equivalent to not displaying bullet comments), and the terminal device enters Figure 8 The interface shown in b in the figure does not display the comment 802. The specific display process can refer to the above Figure 7B , which will not be described in detail here.

[0196] It is understandable that the method for adaptively adjusting the refresh rate provided in the embodiment of the present application can also be applied to other scenarios, such as camera code scanning, game scenarios, video pause scenarios, etc., which are not limited here.

[0197] The method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above method.

[0198] The method provided in the embodiment of the present application can be applied to an electronic device with a display function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be repeated here.

[0199] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0200] For example, Fig. 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Fig. 9 As shown, the terminal device includes: the terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0201] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0202] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0203] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0204] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0205] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0206] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments of the present application, the terminal device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0207] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of a terminal device. While the charging management module 140 is charging the battery 142, it can also power the electronic device through the power management module 141.

[0208] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0209] The terminal device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0210] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0211] The terminal device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0212] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0213] The camera 193 is used to capture static images or videos. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1. The types of the multiple cameras 193 may be different. For example, the camera 193 may include a camera for acquiring color images or a TOF camera, etc. In the embodiment of the present application, the camera may be understood as a camera. The camera includes: an AO camera, a TOF camera, etc. in the embodiment of the present application.

[0214] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0215] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area.

[0216] The terminal device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0217] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The terminal device may receive the button input and generate a key signal input related to the user settings and function control of the terminal device. The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, message, missed call, notification, etc.

[0218] In addition, on top of the above components, the device also runs an operating system, such as an iOS operating system, an Android operating system, or a Windows operating system, etc. Applications can be installed and run on the operating system.

[0219] The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0220] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0221] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0222] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0223] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0224] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0225] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0226] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An interface display method, characterized in that: Applied to a terminal device, the terminal device comprises: a display screen and a display driver; the method comprises: When receiving the image transmitted by the display driver, the display screen refreshes and displays the image transmitted by the display driver; Wherein, when the display screen receives the Nth frame image transmitted by the display driver, it refreshes and displays the Nth frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is the first frame rate; During a first duration of refreshing and displaying the Nth frame image, if the display screen does not receive the N+1th frame image transmitted by the display driver, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate; After a first period of time after the display screen refreshes and displays the Nth frame image, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image, and the corresponding frame rate when the N+1th frame image is drawn and rendered is a second frame rate, and the second frame rate is less than the first frame rate.

2. The method according to claim 1, characterized in that Before the display screen refreshes the display when receiving the image transmitted by the display driver, the method further includes: The terminal device obtains display screen brightness and / or ambient light intensity; The display screen refreshes the display when receiving the image transmitted by the display driver, including: When the brightness of the display screen is greater than a first value, and / or the ambient light intensity is greater than a second value, the display screen refreshes the display when receiving the image transmitted by the display driver.

3. The method according to claim 2, characterized in that The method further comprises: The terminal device determines a first refresh rate based on the brightness of the display screen and / or the ambient light intensity; In a case where the display screen does not receive the N+1th frame image transmitted by the display driver within a third time period after refreshing and displaying the Nth frame image, the display screen reads the Nth frame image from the cache and refreshes and displays the Nth frame image within a third time period after refreshing and displaying the Nth frame image, wherein the third time period is the cycle time period corresponding to the first refresh rate.

4. The method according to claim 3, characterized in that The terminal device stores a first corresponding relationship; the first corresponding relationship is the relationship between the brightness of the display screen and the first refresh rate; The terminal device confirms the first refresh rate based on the brightness of the display screen and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the brightness of the display screen and the first corresponding relationship; Alternatively, the terminal device stores a second corresponding relationship; the second corresponding relationship is the relationship between the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the ambient light intensity and the second corresponding relationship; Alternatively, the terminal device stores a third corresponding relationship; the third corresponding relationship is the relationship between the display screen brightness, the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness, the ambient light intensity and the third corresponding relationship.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the brightness of the display screen is less than or equal to the first value, and the ambient light intensity is less than or equal to the second value, the display screen refreshes the display at a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device; Wherein, when the display screen receives the Mth frame image transmitted by the display driver, it refreshes and displays the Mth frame image, and the frame rate corresponding to the drawing and rendering of the Mth frame image is the third frame rate; In a case where the display screen does not receive the M+1th frame image transmitted by the display driver within a second time duration after refreshing and displaying the Mth frame image, the display screen reads the Mth frame image from the cache and refreshes and displays the Mth frame image during a second time duration after refreshing and displaying the first application interface, and the second time duration is equal to the cycle duration corresponding to the third frame rate; After receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image. The frame rate corresponding to the drawing and rendering of the Nth frame image is the fourth frame rate, and the fourth frame rate is less than the third frame rate.

6. The method according to claims 1-5, characterized in that: The terminal device pre-stores an application list; The application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

7. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image including the bullet comments, and the N+1th frame image is an image not including the bullet comments.

8. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image corresponding to a first scene; the N+1th frame image is an image corresponding to a second scene, the first scene is a battle scene of a game application, and the second scene is a non-battle scene of the game application.

9. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 9.

Images