Interface display method and related device

Abstract

The invention provides an interface display method and a related device. The method comprises the following steps: when a display screen receives an Nth frame of image transmitted by a display driver, refreshing and displaying the Nth frame of image, wherein a frame rate corresponding to the Nth frame of image during drawing and rendering is a first frame rate; in the first duration of refreshing and displaying the Nth frame of image, under the condition that the display screen does not receive the (N + 1) th frame of image transmitted by the display driver, the display screen does not read the Nth frame of image from the cache and keeps displaying the Nth frame of image; the first duration is greater than the period duration corresponding to the first frame rate; after the display screen refreshes and displays the Nth frame of image for a first time length, when the display screen receives the (N + 1) th frame of image transmitted by the display driver, the display screen refreshes and displays the (N + 1) th frame of image, the frame rate corresponding to the (N + 1) th frame of image during drawing and rendering is a second frame rate, and the second frame rate is smaller than the first frame rate. The refresh rate is adaptively adjusted, the image refresh frequency is reduced, and power consumption is reduced.

Description

Interface display method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 12, 2023, with application number 202310862232.7 and application name “Interface Display Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an interface display method and related devices. Background Art

[0003] Currently, terminal devices can support a variety of screen refresh rates, such as 60 times per second (Hz), 90Hz, 120Hz, or 144Hz. The screen refresh rate may or may not be consistent with the application frame rate.

[0004] How to reduce the power consumption of terminal devices based on the relationship between screen refresh rate and application frame rate is a problem that needs to be solved.

[0005] Summary of the Invention

[0006] The present invention provides an interface display method and related device for use in the field of terminal technology. The display driver chip of a display screen can adaptively adjust the refresh rate. When the display driver is not sending an image for display, the image is not read from the cache for image refresh. Image refresh is not performed until a preset time has elapsed or an image is sent for display. This method of refreshing the image based on the image sending time can reduce the number of image refreshes when no new composite image is being sent for display, thereby reducing the power consumption of the display screen.

[0007] In a first aspect, an embodiment of the present application proposes an interface display method. The method includes: when the display screen receives the image transmitted by the display driver, refreshing and displaying the image transmitted by the display driver; wherein, when the display screen receives the Nth frame image transmitted by the display driver, refreshing and displaying the Nth frame image, the frame rate corresponding to the drawing and rendering of the Nth frame image is a first frame rate; within a first duration of refreshing and displaying the Nth frame image, if the display screen does not receive the N+1th frame image transmitted by the display driver, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate; after a first duration after the display screen refreshes and displays the Nth frame image, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image, the frame rate corresponding to the drawing and rendering of the N+1 frame image is a second frame rate, and the second frame rate is less than the first frame rate.

[0008] This way, when the application frame rate decreases, the display's refresh rate is adjusted to a lower level, reducing the display's image refresh frequency and lowering power consumption. The terminal device can adaptively adjust the refresh rate, setting a higher refresh rate in some scenarios to meet smoothness requirements and a lower refresh rate in others to reduce power consumption. Furthermore, by adjusting the display's refresh rate based on the image being displayed, there's no need to identify the application's frame rate. This method can be used in low-frame-rate scenarios such as video and camera applications, covering a wide range of scenarios and offering high practicality.

[0009] Optionally, before the display screen refreshes the display when receiving the image transmitted by the display driver, the method also includes: the terminal device obtains the display screen brightness, and / or the ambient light intensity; the display screen refreshes the display when receiving the image transmitted by the display driver, including: when the display screen brightness is greater than a first value, and / or the ambient light intensity is greater than a second value, the display screen refreshes the display when receiving the image transmitted by the display driver.

[0010] In this way, the terminal device can achieve adaptive refresh in scenarios where the display brightness and / or ambient light intensity meet the conditions, thereby reducing power consumption; in other scenarios, adaptive refresh is not used to reduce problems such as screen flickering caused by refresh rate switching.

[0011] Optionally, the method also includes: the terminal device determines a first refresh rate based on the brightness of the display screen and / or the ambient light intensity; when the display screen does not receive the N+1th frame image transmitted by the display driver within a third time period after refreshing and displaying the Nth frame image, the display screen reads the Nth frame image from the cache and refreshes and displays the Nth frame image within a third time period after refreshing and displaying the Nth frame image, wherein the third time period is the cycle time period corresponding to the first refresh rate.

[0012] In this way, the minimum refresh rate is set based on the display brightness and / or ambient light intensity, which can reduce energy consumption as much as possible while reducing screen flickering of the terminal device. This can also reduce image refresh display flickering caused by excessive refresh time during adaptive refresh.

[0013] Optionally, the terminal device stores a first correspondence; the first correspondence is the relationship between the display screen brightness and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness and the first correspondence; or, the terminal device stores a second correspondence; the second correspondence is the relationship between the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the ambient light intensity and the second correspondence; or, the terminal device stores a third correspondence; the third correspondence is the relationship between the display screen brightness, the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness, the ambient light intensity and the third correspondence.

[0014] In this way, it is convenient for the terminal device to confirm the minimum refresh rate, which is simple and easy to implement.

[0015] Optionally, the method further includes: when the brightness of the display screen is less than or equal to the first value, and the ambient light intensity is less than or equal to the second value, the display screen refreshes the display at a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device; wherein, upon receiving the Mth frame image transmitted by the display driver, the display screen refreshes and displays the Mth frame image, and the frame rate corresponding to the drawing and rendering of the Mth frame image is a third frame rate; within a second time period after refreshing and displaying the Mth frame image, if the display screen does not receive the M+1th frame image transmitted by the display driver, the display screen reads the Mth frame image from the cache and refreshes and displays the Mth frame image within a second time period after refreshing and displaying the first application interface, and the second time period is equal to the cycle time corresponding to the third frame rate; after receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is a fourth frame rate, and the fourth frame rate is less than the third frame rate.

[0016] In this way, the terminal device can adapt to the lower refresh rate of the display when the application frame rate becomes smaller, reduce the image refresh frequency of the display, and reduce power consumption.

[0017] Optionally, the terminal device pre-stores an application list; the application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

[0018] In this way, the terminal device can confirm whether to perform adaptive refresh based on the application.

[0019] Optionally, the Nth frame image is an image including barrage, and the N+1th frame image is an image not including barrage.

[0020] In this way, when there is a barrage, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there is no barrage, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0021] Optionally, the Nth frame image is an image corresponding to a first scene; the N+1th frame image is an image corresponding to a second scene, the first scene is a battle scene of a game application, and the second scene is a non-battle scene of the game application.

[0022] In this way, in scenes with high requirements for smoothness, such as game battle scenes, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; in scenes with lower requirements for smoothness, such as game lobby scenes, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0023] Optionally, the Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

[0024] In this way, when there are motion effects, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there are no motion effects, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0025] In a second aspect, embodiments of the present application provide a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver functions, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc.

[0026] The terminal device includes:

[0027] The terminal device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of the first aspect.

[0028] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is run, it enables the computer to execute the method of the first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method of the fourth aspect.

[0031] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram showing a display flow chart of a possible design;

[0033] FIG2 is a schematic diagram of a display process provided by an embodiment of the present application;

[0034] FIG3 is a schematic diagram of the structure of a software framework of a terminal device provided in an embodiment of the present application;

[0035] FIG4 is a schematic diagram of a flow chart of internal module interactions corresponding to an interface display method provided in an embodiment of the present application;

[0036] FIG5 is a schematic diagram of a display process provided by an embodiment of the present application;

[0037] FIG6 is a flow chart of an interface display method provided in an embodiment of the present application;

[0038] FIG7A is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0039] FIG7B is a schematic diagram of a display process provided by an embodiment of the present application;

[0040] FIG8 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0041] FIG9 is a schematic structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] To facilitate understanding of the embodiments of the present application, some of the terms involved in the present application are first briefly explained.

[0043] 1. Frame: This refers to the smallest unit of a single image in an interface display. A frame can be thought of as a still image. Displaying multiple frames in rapid succession can create the illusion of motion. Frame rate refers to the number of frames refreshed per second, or the number of times per second the graphics processor in a terminal device refreshes the image. A higher frame rate results in smoother and more realistic animation. The more frames per second, the smoother the displayed motion.

[0044] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0045] 2. Drawing: This refers to the image drawing of the display interface. The display interface can be composed of one or more views. Each view can be drawn by the visual controls of the view system. Each view is composed of subviews. A subview corresponds to a small widget in the view. For example, a subview corresponds to a symbol in the image view.

[0046] 3. Rendering: It is to color the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0047] 4. Synthesis: It is the process of synthesizing multiple or more rendered views into a display interface.

[0048] 5. Other terms

[0049] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the terms "first chip" and "second chip" are used solely to distinguish between different chips and do not define their order. Those skilled in the art will understand that terms such as "first" and "second" do not define the quantity or execution order, and do not necessarily define differences.

[0050] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0051] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0052] 6. Terminal equipment

[0053] The terminal device of the embodiment of the present application may also be any form of electronic device. For example, the electronic device may include a handheld device, a vehicle-mounted device, etc. For example, some electronic devices include: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (PLMNs), and the like. The terminal equipment in the network (PLMN), etc., is not limited to this in the embodiments of the present application.

[0054] As an example and not a limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0055] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0056] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0057] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also known as main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as the Linux operating system, the Unix operating system, the Android operating system, the iOS operating system, or the Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0058] Currently, terminal devices can support a variety of screen refresh rates, such as 60 times per second (Hz), 90Hz, 120Hz, or 144Hz. The screen refresh rate may or may not be consistent with the application frame rate.

[0059] In scenarios with low frame rates, such as video and camera applications, content such as bullet comments and animations may need to be displayed. To ensure smooth display of bullet comments and animations, the screen refresh rate is usually higher than the application frame rate in scenarios with low frame rates, such as video and camera applications.

[0060] In a possible design, during video playback, if a bullet screen is displayed, the terminal device synthesizes and displays the image at a frequency of 60 frames per second to improve the smoothness of the bullet screen display. When the bullet screen disappears, the terminal device synthesizes the image at a frequency of 30 frames per second, while the terminal device continues to refresh the displayed image at a frequency of 60 frames per second, which consumes a lot of power. In addition, when the terminal device pauses the video, the terminal device's display screen will continue to refresh the displayed image at a frequency of 60 frames per second, which consumes a lot of power.

[0061] For example, Figure 1 shows a display flow diagram for a possible design. For example, if a video is played at 30 frames per second and the bullet comments are displayed at 60 frames per second, the video in Figure 1 corresponds to frames 1, 2, and 3. The bullet comments correspond to bullet comments 1, 2, and 3. Since the bullet comments are displayed at 60 frames per second, the display refreshes at 60 frames per second.

[0062] At 0ms, the image synthesis service (surface flinger) in the terminal device synthesizes frame 1 and barrage 1.

[0063] At 16.6ms, the image synthesis service synthesizes frame 1 and bullet comment 2, and the display driver transmits the synthesized frame 1 and bullet comment 1 to the display screen for display. The display screen begins to display frame 1 and bullet comment 1.

[0064] At 33.2ms, the image synthesis service synthesizes frame 2 and bullet comment 3, and the display driver transmits the synthesized frame 1 and bullet comment 2 to the display screen for display. The display screen begins to display frame 1 and bullet comment 2.

[0065] At 49.8ms, the bullet comment display is complete. The image synthesis service does not synthesize the image. The display driver transmits the synthesized frame 2 and bullet comment 3 to the display screen for display. The display screen starts to display frame 2 and bullet comment 3.

[0066] At 64.4ms, the image synthesis service synthesizes frame 3. Because the image synthesis service did not perform the synthesis operation at 49.8ms, the display driver did not transmit the synthesized image to the display. The display reads display frame 2 and bullet comment 3 from the cache and refreshes the display frame 2 and bullet comment 3.

[0067] At 83.0ms, the image composition service does not perform composition, and the display driver transmits the composed frame 3 to the display. The display refreshes and displays frame 3.

[0068] At 99.6ms, the Image Composition Service composites frame 4. Because the Image Composition Service did not perform the composite operation at 83.0ms, the display driver does not transmit the composited image to the display. The display reads frame 3 from the cache and refreshes the display.

[0069] At 116.2ms, the image composition service does not perform composition, and the display driver transmits the composed frame 4 to the display. The display refreshes and displays frame 4.

[0070] As can be seen from Figure 1, at 64.4ms and 99.6ms, when there is no new image to display, the terminal device's display screen will still read the image from the display screen's cache and refresh the displayed image. This results in higher power consumption for the terminal device.

[0071] In light of this, embodiments of the present application provide a refresh rate switching method and related apparatus. The display driver chip of a display screen can adaptively adjust the refresh rate. When the display driver is not sending an image for display, the image is not read from the cache for image refresh. Image refresh is not performed until a preset time has elapsed or an image is sent for display. In this way, image refresh based on the image sending time can reduce the number of image refreshes when there is no new composite image to be sent for display, thereby reducing the power consumption of the display screen.

[0072] The display process of adaptive refresh is described below with reference to FIG2 .

[0073] For example, Figure 2 is a schematic diagram of a display process provided by an embodiment of the present application. Taking the video playing at 30 frames per second and the bullet screen displaying at 60 frames per second as an example, in Figure 2, the video corresponds to frames 1, 2, and 3. The bullet screen corresponds to bullet screen 1, 2, and 3. Since the bullet screen is displayed at 60 frames per second, the display screen refreshes at 60 frames per second.

[0074] In Figure 2, the video corresponds to frames 1, 2, and 3. The bullet comments correspond to bullet comments 1, 2, and 3. Since the bullet comments are displayed at 60 frames per second, the display screen refreshes at 60 frames per second.

[0075] At 0ms, the image synthesis service (surface flinger) in the terminal device synthesizes frame 1 and barrage 1.

[0076] At 16.6ms, the image synthesis service synthesizes frame 1 and bullet comment 2, and the display driver transmits the synthesized frame 1 and bullet comment 1 to the display screen for display. The display screen begins to display frame 1 and bullet comment 1.

[0077] At 33.2ms, the image synthesis service synthesizes frame 2 and bullet comment 3, and the display driver transmits the synthesized frame 1 and bullet comment 2 to the display screen for display. The display screen begins to display frame 1 and bullet comment 2.

[0078] At 49.8ms, the bullet comment display is complete. The image synthesis service does not synthesize the image. The display driver transmits the synthesized frame 2 and bullet comment 3 to the display screen for display. The display screen starts to display frame 2 and bullet comment 3.

[0079] At 64.4ms, the image synthesis service synthesizes frame 3. Because the image synthesis service did not perform synthesis at 49.8ms, the display driver does not transmit the synthesized image to the display. The display does not read frame 2 and bullet comment 3 from the cache, does not refresh the image, and continues to display frame 2 and bullet comment 3.

[0080] At 83.0ms, the image composition service does not perform composition, and the display driver transmits the composed frame 3 to the display. The display refreshes and displays frame 3.

[0081] At 99.6ms, the Image Composition Service synthesizes frame 4. Because the Image Composition Service did not perform the synthesis operation at 83.0ms, the display driver does not transmit the synthesized image to the display. The display does not read frame 3 from the cache, does not refresh the image, and keeps displaying frame 3.

[0082] At 116.2ms, the image composition service does not perform composition, and the display driver transmits the composed frame 4 to the display. The display refreshes and displays frame 4.

[0083] As shown in Figure 2, at 64.4ms and 99.6ms, when no image is being sent to the terminal device's display, the display does not read the image from the cache and does not refresh the image. Compared to the display process shown in Figure 1, the process shown in Figure 2 has fewer display image refreshes, which can reduce the power consumption of the terminal device.

[0084] The method provided in the embodiment of the present application is described below with reference to FIG. 3 to FIG. 7B .

[0085] For ease of understanding, the following describes the software framework of the terminal device. The terminal device software system can adopt a layered architecture, event-driven architecture, micro-kernel architecture, microservice architecture, or cloud architecture.

[0086] Taking the layered architecture of the Android system as an example, the software structure of the terminal device is exemplified. For example, FIG3 is a schematic diagram of the structure of the software framework of a terminal device provided in an embodiment of the present application. As shown in FIG3, the layered architecture divides the software system of the terminal device into several layers, each layer having a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, namely, the application layer (applications), the application framework layer (application framework), the hardware abstract layer (HAL), the kernel layer (kernel) and the hardware layer.

[0087] The application layer includes a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. As shown in Figure 3, the application package may include applications such as camera, calendar, map, phone, music, settings, notes, and video.

[0088] The application framework layer provides an API and programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 3, the application framework layer may include a window manager, image composition service, frame rate control service, content provider, telephony manager, resource manager, notification manager, and so on.

[0089] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0090] The image composition service is used to compose images and transmit the composed images to the display driver to display the images on the display screen. Exemplarily, the image composition service can compose images rendered by one or more applications.

[0091] The frame rate control service is used to adjust the refresh rate of the display screen through the display driver. In some embodiments, the frame rate control service includes: a brightness ambient light processing module. The brightness ambient light processing module is used to control the adaptive switch in the display driver to be turned on or off based on the brightness of the display screen and the ambient light intensity of the terminal device to adjust the refresh strategy of the display screen. When the brightness of the display screen is greater than the first value and the ambient light intensity is greater than the second value, the brightness ambient light processing module controls the adaptive switch in the display driver to be turned on to achieve adaptive refresh of the display screen; when the brightness of the display screen is less than or equal to the first value, or the ambient light intensity is less than or equal to the second value, the brightness ambient light processing module controls the adaptive switch in the display driver to be turned off, and the display screen is refreshed at a certain frequency.

[0092] The sensor service is used to obtain the display brightness and ambient light intensity of the terminal device.

[0093] Content providers store and retrieve data and make it accessible to applications. This data can include video, images, audio, incoming and outgoing calls, browsing history and bookmarks, and phone books. The view system includes visual controls, such as those that display text and images.

[0094] The phone manager is used to provide communication functions for terminal devices, such as call status management (including answering, hanging up, etc.).

[0095] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc.

[0096] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the device, or flashing indicator lights.

[0097] The hardware abstraction layer can contain multiple library modules, such as hardware synthesizers. The Android system can load the corresponding library modules for the device hardware, thereby enabling the application framework layer to access the device hardware. Device hardware can include, for example, the display screen in the terminal device.

[0098] The kernel layer is the layer between hardware and software. The kernel layer is used to drive the hardware, making it work. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, etc., which are not limited in this embodiment of the application.

[0099] In some embodiments, the display driver includes an adaptive switch that controls the refresh strategy of the display screen. When the adaptive switch is on, the display screen does not refresh when the display driver is not displaying an image, until a preset duration has elapsed or the display driver displays an image. When the adaptive switch is off, the display screen refreshes at a predetermined frequency, even when the display driver is not displaying an image.

[0100] The display process of the terminal device is described below with reference to FIG3 .

[0101] Applications use the view system in the application framework layer to call the system's image rendering library to render images and transfer the rendered images to the cache queue of the image compositing service. The image compositing service composites the rendered images of one or more applications and transfers the composited image to the display driver. The display driver then transmits the composited image to the display screen for display.

[0102] For example, FIG4 is a flow chart of the internal module interaction corresponding to an interface display method provided in an embodiment of the present application. The terminal device includes: a frame rate control service, a display driver, an application, an image synthesis service, and a display screen. As shown in FIG4, the interaction process includes:

[0103] S401. After the device is powered on, the frame rate control service registers a callback with the sensor service to obtain the display brightness of the terminal device and / or the ambient light intensity of the terminal device.

[0104] In some embodiments, after powering on, the frame rate control service registers a callback with the sensor service to facilitate subsequent updating of the display brightness and / or the ambient light intensity when the display brightness of the terminal device changes or the ambient light intensity of the terminal device changes, or obtaining display brightness change information and / or ambient light intensity change information.

[0105] In other embodiments, the frame rate control service may also obtain the display brightness and / or ambient light intensity of the terminal device from the sensor service at a preset time interval. The embodiments of the present application do not limit the manner in which the frame rate control service obtains the display brightness and ambient light intensity.

[0106] S402: The sensor service transmits display screen brightness information and / or ambient light intensity information to the frame rate control service.

[0107] The display brightness information is used to indicate the brightness of the display or to indicate changes in the brightness of the display. The ambient light intensity information is used to indicate the ambient light intensity or to indicate changes in the ambient light intensity.

[0108] The frame rate control service determines the refresh strategy of the terminal device's display based on the display brightness, ambient light intensity, and preset conditions.

[0109] When the display brightness is greater than the first value and / or the ambient light intensity is greater than the second value, the terminal device executes S403-S405. When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the terminal device executes S406-S405.

[0110] S403: When the display screen brightness is greater than a first value and / or the ambient light intensity is greater than a second value, the frame rate control service transmits a message to the display driver for instructing the adaptive switch to be turned on.

[0111] The first value can be 50 nit, 60 nit, or any other value. The second value can be 50 lux, 60 lux, or any other value. The specific values ​​of the first and second values ​​are not limited in this embodiment of the application.

[0112] It should be noted that the human eye perceives different aspects of a terminal device under different display brightness and ambient light intensities. If the light is dim or the display brightness is low, the human eye may perceive flicker when the terminal device switches the refresh rate, which may reduce the user experience.

[0113] It is understood that the terminal device may determine whether the adaptive switch is on or off based solely on the display brightness; the terminal device may also determine whether the adaptive switch is on or off based solely on the ambient light intensity; or the terminal device may further determine whether the adaptive switch is on or off based on both the display brightness and the ambient light intensity. This embodiment of the present application is not limited to this.

[0114] In some embodiments, the frame rate control service transmits a message to the display driver through an interface.

[0115] S404 : After receiving the message for instructing the adaptive switch to be turned on, the display driver transmits a message for instructing the adaptive refresh to the display screen.

[0116] S405 . When the display screen receives the synthesized image transmitted by the display driver, it refreshes and displays the synthesized image.

[0117] In the embodiment of the present application, the display screen includes a display driver IC (DDIC), which can control the refresh of the display screen of the terminal device and the refresh interval.

[0118] S406: When the brightness of the display screen is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the frame rate control service transmits a message indicating a preset refresh rate to the display driver.

[0119] S407 : After receiving the message indicating the preset refresh rate, the display driver transmits the message indicating the preset refresh rate to the display screen.

[0120] S408: The display screen refreshes and displays the synthesized image according to a preset refresh rate.

[0121] The preset refresh rate can be 60 Hz, 120 Hz, or any other value, which is not limited in the embodiment of the present application.

[0122] In a possible implementation, the preset refresh rate is related to the foreground application of the terminal device. When the foreground application is a video application, the preset refresh rate may be 60Hz, and when the foreground application is a game application, the preset refresh rate may be 120Hz.

[0123] The preset refresh rate may also be related to the operating load of the terminal device, etc. The embodiment of the present application does not limit the method for determining the preset refresh rate.

[0124] In this way, the terminal device can control adaptive refresh when the display brightness is high and / or the ambient light intensity is high, reducing the refresh frequency of the display and reducing power consumption. The terminal device can control the terminal device to refresh at a preset refresh rate when the display brightness is low and the ambient light intensity is low, reducing problems such as screen flickering caused by refresh rate switching.

[0125] The two refresh strategies provided in FIG4 are described below in conjunction with FIG5. For example, the image synthesis service (surface flinger) synthesizes frames 1, 2, 3, 4, and 5 at 0ms, 32.2ms, 49.8ms, 64.4ms, and 99.6ms, respectively.

[0126] When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the display process of the terminal device may be as shown in a of Figure 5. Specifically, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display driver transmits synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5 to the display screen, respectively. Adaptively, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display screen begins to display synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5, respectively.

[0127] The display screen does not receive the synthesized frame transmitted by the display driver at 32.2ms, reads the synthesized frame 1 from the cache, refreshes, and displays the synthesized frame 1. The display screen does not receive the synthesized frame transmitted by the display driver at 99.6ms, reads the synthesized frame 4 from the cache, and refreshes to continue displaying the synthesized frame 4.

[0128] In this way, the refresh rate of the display is fixed, which can reduce problems such as screen flickering caused by refresh rate switching and improve user experience.

[0129] When the display brightness is greater than the first value or the ambient light intensity is greater than the second value, the display process of the terminal device may be as shown in a of Figure 5. Specifically, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display driver transmits synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5 to the display screen, respectively. Adaptively, at 16.6ms, 49.8ms, 64.4ms, 83.0ms, and 116.2ms, the display screen begins to display synthesized frame 1, synthesized frame 2, synthesized frame 3, synthesized frame 4, and synthesized frame 5, respectively.

[0130] The display screen does not receive the synthesized frame transmitted by the display driver at 32.2ms, does not read the synthesized frame from the cache, does not refresh the image, and keeps displaying synthesized frame 1. The display screen does not receive the synthesized frame transmitted by the display driver at 99.6ms, does not read the synthesized frame from the cache, does not refresh the image, and keeps displaying synthesized frame 4.

[0131] Compared with the process shown in a in FIG. 5 , the process shown in b in FIG. 5 can reduce two image refreshes and reduce the power consumption of the terminal device.

[0132] It's important to note that after a display refreshes its image, the voltage across the light-emitting devices decays over time, reducing the image's brightness. When the terminal device refreshes the image again, the voltage across the light-emitting devices increases. If the image hasn't been refreshed for a long time, the voltage difference between the light-emitting devices before and after the refresh is significant, leading to significant differences in brightness. This can cause the human eye to perceive anomalies such as screen flickering, which in turn reduces the user experience of the terminal device.

[0133] Based on the above embodiment, the message in S403 indicating the activation of the adaptive switch includes the minimum refresh rate. The message in S407 indicating the adaptive refresh includes the minimum refresh rate. Adaptively, S405 further includes: when the display screen does not receive the synthesized image within a first preset time duration after refreshing the image, reading the frame image from the cache, performing an image refresh, and displaying the frame image. The first preset time duration is related to the minimum refresh rate. The first preset time duration may be the reciprocal of the minimum refresh rate.

[0134] The minimum refresh rate can be 10 Hz, 30 Hz, or any other value, which is not limited in the embodiments of the present application.

[0135] In this way, limiting the minimum refresh rate of adaptive refresh can reduce abnormal phenomena such as screen flickering on terminal devices and improve user experience.

[0136] In the embodiment of the present application, the refresh rate of the display screen can be 1 Hz, 10 Hz, 30 Hz, 40 Hz, 60 Hz, 120 Hz, etc. The embodiment of the present application does not limit the specific value of the refresh rate of the display screen.

[0137] In some embodiments, the frame rate control service may also determine the minimum refresh rate based on display brightness and / or ambient light intensity.

[0138] In a possible implementation manner 1, the frame rate control service stores a first correspondence relationship, which is a relationship between display brightness and minimum refresh rate.

[0139] For example, the display brightness is 50 nits and the minimum refresh rate is 10 Hz; the display brightness is 60 nits and the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the first corresponding relationship.

[0140] In this way, the minimum refresh rate is set based on the brightness of the display screen, which can reduce energy consumption as much as possible while reducing screen flickering on the terminal device.

[0141] In a second possible implementation, the frame rate control service stores a second corresponding relationship, which is a relationship between ambient light intensity and the minimum refresh rate.

[0142] For example, when the ambient light intensity is 50 lux, the minimum refresh rate is 10 Hz; when the ambient light intensity is 60 lux, the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the second corresponding relationship.

[0143] In this way, the minimum refresh rate is set based on the ambient light intensity, which can reduce energy consumption as much as possible while reducing the screen flicker of the terminal device.

[0144] In possible implementation manner three, the frame rate control service stores a third corresponding relationship, which is a relationship between display brightness, ambient light intensity, and minimum refresh rate.

[0145] For example, when the display brightness is 50 nits and the ambient light intensity is 50 lux, the minimum refresh rate is 10 Hz; when the display brightness is 60 nits and the ambient light intensity is 60 lux, the minimum refresh rate is 30 Hz. The embodiment of the present application does not specifically limit the specific value of the third corresponding relationship.

[0146] In this way, the minimum refresh rate is set based on the display brightness and ambient light intensity, which can reduce energy consumption as much as possible while reducing screen flickering on terminal devices.

[0147] Based on the above embodiment, the frame rate control service can also determine the minimum refresh rate based on the highest frame rate corresponding to the foreground running application. Under the same conditions, the higher the highest frame rate corresponding to the foreground running application, the higher the minimum refresh rate.

[0148] In this way, the refresh time difference and refresh rate difference during adaptive refresh can be reduced, and the flicker during frame rate switching can be further reduced.

[0149] Based on the above embodiment, the activation of the adaptive refresh is also related to the foreground running application. Alternatively, it can be understood that the terminal device can control the display screen to adaptively refresh based on the specific application.

[0150] In some embodiments, the frame rate control service stores an application list. When the foreground application is one of the applications in the application list, the frame rate control service determines whether the adaptive switch is on or off based on the display brightness and / or ambient light intensity. When the foreground application is not one of the applications in the application list, the frame rate control service transmits a message to the display driver indicating a preset refresh rate.

[0151] For example, FIG6 is a flow chart of an interface display method provided in an embodiment of the present application. As shown in FIG6, the method includes:

[0152] S601: When receiving an image from a display driver, the display screen of the terminal device refreshes and displays the image transmitted by the display driver.

[0153] S602 : When the display screen receives the Nth frame image from the display driver, it refreshes and displays the Nth frame image. The frame rate corresponding to the rendering of the Nth frame image is the first frame rate.

[0154] S603: If the display screen does not receive the (N+1)th frame image transmitted by the display driver within the first duration for refreshing the display of the Nth frame image, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate;

[0155] S604. After the display screen refreshes and displays the Nth frame image for a first period of time, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image. The corresponding frame rate when the N+1th frame image is drawn and rendered is the second frame rate, and the second frame rate is less than the first frame rate.

[0156] It is understandable that the first frame rate can be 60 frames / second, 120 frames / second or any other value, and the second frame rate can be 30 frames / second, 60 frames / second or any other value, and the implementation of this application does not limit this.

[0157] In this way, the terminal device can adapt to the lower refresh rate of the display when the application frame rate becomes smaller, reduce the image refresh frequency of the display, and reduce power consumption.

[0158] Furthermore, the refresh rate of the display can be adjusted based on the image being fed to the display, eliminating the need to identify the application's frame rate. This approach can be used in low-frame-rate scenarios like video and camera feeds, covering a wide range of scenarios and offering high practicality.

[0159] Optionally, the terminal device pre-stores an application list; the application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

[0160] In this way, the terminal device can confirm whether to perform adaptive refresh based on the application.

[0161] Optionally, the Nth frame image is an image including the barrage, and the N+1th frame image is an image not including the barrage.

[0162] In this way, when there is a barrage, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there is no barrage, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0163] Optionally, the Nth frame image is an image corresponding to the first scene; the N+1th frame image is an image corresponding to the second scene, the first scene is a battle scene of the game application, and the second scene is a non-battle scene of the game application.

[0164] In this way, in scenes with high requirements for smoothness, such as game battle scenes, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; in scenes with lower requirements for smoothness, such as game lobby scenes, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0165] Optionally, the Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

[0166] In this way, when there are motion effects, the refresh rate of the display screen is higher to ensure the smoothness of the barrage display; when there are no motion effects, the refresh rate of the display screen is reduced to reduce the power consumption of the terminal device and improve the user experience.

[0167] Based on the above embodiment, the terminal device can also confirm that the display screen is adaptively refreshed or refreshed at a certain refresh rate based on the display screen brightness and / or the ambient light intensity, and confirm the execution of S601-S604 or the execution of the following S605-S608.

[0168] In this way, the terminal device can achieve adaptive refresh in some scenarios to reduce power consumption; in other scenarios, it can refresh at a certain refresh rate to reduce problems such as screen flickering caused by refresh rate switching.

[0169] In a possible implementation mode 1, when the display screen brightness is greater than the first value, the terminal device executes S601 to S604. When the display screen brightness is less than or equal to the first value, the terminal device executes S605 to S608.

[0170] S605: The display screen refreshes the display according to a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device;

[0171] S606: When the display screen receives the M-th frame image transmitted by the display driver, it refreshes and displays the M-th frame image, and the frame rate corresponding to the rendering of the M-th frame image is the third frame rate;

[0172] S607: If the display screen does not receive the M+1th frame image transmitted by the display driver within a second time duration after refreshing the display of the Mth frame image, the display screen reads the Mth frame image from the cache and refreshes and displays the Mth frame image during a second time duration after refreshing the display of the first application interface, where the second time duration is equal to the cycle duration corresponding to the third frame rate;

[0173] S608. After receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image. The frame rate corresponding to the rendering of the Nth frame image is the fourth frame rate, and the fourth frame rate is less than the third frame rate. It is understood that the third frame rate can be 60 frames per second, 120 frames per second, or any other value, and the fourth frame rate can be 30 frames per second, 60 frames per second, or any other value, and this application does not limit this.

[0174] In this way, the terminal device can control adaptive refresh when the display brightness is high, reduce the refresh frequency of the display, and reduce power consumption. When the display brightness is low, the terminal device can control the terminal device to refresh at a preset refresh rate, reducing problems such as screen flickering caused by refresh rate switching.

[0175] In a second possible implementation, when the ambient light intensity is greater than the second value, the terminal device executes S601 to S604. When the ambient light intensity is less than or equal to the second value, the terminal device executes S605 to S608.

[0176] In this way, the terminal device can control adaptive refresh when the ambient light intensity is high, reduce the refresh frequency of the display screen, and reduce power consumption. When the ambient light intensity is low, the terminal device can control the terminal device to refresh at a preset refresh rate, reducing problems such as screen flickering caused by refresh rate switching.

[0177] In possible implementation mode 3, when the display brightness is greater than the first value or the ambient light intensity is greater than the second value, the terminal device executes S601-S604. When the display brightness is less than or equal to the first value and the ambient light intensity is less than or equal to the second value, the terminal device executes S605-S608.

[0178] In this way, the terminal device can control adaptive refresh when the display brightness is high or the ambient light intensity is high, reducing the refresh frequency of the display and reducing power consumption. The terminal device can control the terminal device to refresh at a preset refresh rate when the display brightness is low and the ambient light intensity is low, reducing problems such as screen flickering caused by refresh rate switching.

[0179] In possible implementation mode 4, when the display brightness is greater than the first value and the ambient light intensity is greater than the second value, the terminal device executes S601-S604. When the display brightness is less than or equal to the first value, or the ambient light intensity is less than or equal to the second value, the terminal device executes S605-S608.

[0180] In this way, the terminal device can control adaptive refresh when the display brightness is high and the ambient light intensity is high, reducing the refresh frequency of the display and reducing power consumption. The terminal device can control the terminal device to refresh at a preset refresh rate when the display brightness is low or the ambient light intensity is low, reducing problems such as screen flickering caused by refresh rate switching.

[0181] The terminal device can confirm that the display screen is adaptively refreshed or refreshed at a preset frequency by any of the possible implementation methods 1 to 4 above. This is not limited here.

[0182] The adaptive refresh after meeting preset conditions provided by the embodiment of the present application is described below in conjunction with specific application scenarios.

[0183] For example, FIG7A is a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario is a video playback scenario. As shown in a of FIG7A , in this application scenario, the terminal device displays a first video 701. During the playback of the first video 701, the terminal device may display a barrage 702 (as shown in b of FIG7A ).

[0184] It is understandable that when the video is playing, the video application draws and renders the video frame at the first frame rate, and the terminal device synthesizes the video frame at the first frame rate. When there is a bullet comment to be displayed, the terminal device synthesizes the video frame and the bullet comment at the second frame rate.

[0185] For example, Figure 7B is a schematic diagram of a display process provided by an embodiment of the present application. The video application renders video frames at 30 frames per second. The terminal device renders bullet comments at 60 frames per second. As shown in Figure 7B, the video frames correspond to frames 1, 2, 3, and 4; and the bullet comments correspond to bullet comments 1, 2, and 3.

[0186] As shown in FIG7B , at 0 ms, the video application draws and renders frame 1.

[0187] At 16.6ms, the video application does not perform drawing or rendering. Since the drawing and rendering of frame 1 and bullet comment 1 are completed, the image synthesis service synthesizes frame 1 and bullet comment 1.

[0188] At 33.2ms, the video application renders frame 2. Because frame 2 is not yet rendered, bullet comment 2 is rendered. The image compositing service then combines frame 1 and bullet comment 2. Since the composition of frame 1 and bullet comment 1 is complete, the display driver transmits the combined frame 1 and bullet comment 1 to the display for display. The display then begins displaying frame 1 and bullet comment 1.

[0189] At 49.8ms, the video application does not perform any rendering. Since frame 2 and bullet comment 3 have both been rendered, the image synthesis service synthesizes frame 2 and bullet comment 3. Since the synthesis of frame 1 and bullet comment 2 is complete, the display driver transmits the synthesized frame 1 and bullet comment 2 to the display for display. The display then begins displaying frame 1 and bullet comment 2.

[0190] At 64.4ms, the video application renders frame 3. Because frame 3 is not yet rendered and no bullet comment has been rendered, the compositing service has no frames to composite and does not perform compositing. Since the composition of frame 2 and bullet comment 3 is complete, the display driver transmits the composited frame 2 and bullet comment 3 to the display for display. The display begins displaying frame 2 and bullet comment 3.

[0191] Because the 49.8ms image synthesis service did not perform the synthesis operation, the display driver did not transmit the synthesized image to the display. The display did not refresh and continued to display frame 2 and bullet screen 3.

[0192] At 83.0ms, the video application does not perform any rendering. Since frame 3 is rendered, the image compositing service synthesizes frame 3. Since the image compositing service does not perform any compositing operations at 64.4ms, the display driver does not transmit any image to the display. The display does not refresh and continues to display frame 2 and bullet comment 3.

[0193] At 99.6ms, the video application renders frame 4. Because frame 4 is not yet rendered and the bullet screen rendering is not complete, the compositing service has no frames to composite and does not perform compositing. Since compositing of frame 3 is complete, the display driver transmits the composited frame 3 to the display for display. The display refreshes and begins displaying frame 3.

[0194] At 116.2ms, the video application does not perform any rendering. Since frame 4 is rendered, the image compositing service composites frame 4. Since the image compositing service does not composite at 99.6ms, the display driver does not transmit any image to the display, and the display does not refresh and continues to display frame 3.

[0195] As can be seen from Figure 7B, the terminal device adaptively adjusts the refresh rate based on whether the barrage is displayed or not. Specifically, the display screen refreshes the barrage at 32.2ms, 49.8ms, and 64.4ms. The time interval for barrage display is 16.6ms. When the barrage refresh display ends, the display screen refreshes the display frame 3 at 99.6ms, and the time interval for video frame refresh display is 32.2ms. Or it can be understood that the corresponding refresh rate when the barrage is refreshed is 30Hz, and the corresponding refresh rate of the video frame when there is no barrage refresh display is 60Hz.

[0196] This allows the terminal device to adaptively adjust its refresh rate based on whether or not the bullet screen is displayed, reducing power consumption. Furthermore, there's no need to adjust the frame rate of the image synthesis service or the frame rate of the application rendering the image, making it easy to implement and providing a fast refresh rate response.

[0197] For example, FIG8 is a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario is a live broadcast scenario. As shown in a in FIG8 , in this application scenario, the terminal device displays a live video 801 and comments 802 (equivalent to displaying a barrage). When the user's finger slides to the right on the display screen, the terminal device receives an operation for instructing to clear the screen (equivalent to not displaying the barrage after clearing the screen), and the terminal device enters the interface shown in b in FIG8 , which does not display comments 802. The specific display process can be referred to FIG7B above, which will not be described in detail here.

[0198] It is understandable that the method for adaptively adjusting the refresh rate provided in the embodiments of the present application can also be applied to other scenarios, such as camera code scanning, gaming, and video pause scenarios, etc. This is not limited here.

[0199] The method of the embodiment of the present application has been described above. The device for performing the above method provided by the embodiment of the present application is described below. Those skilled in the art will understand that the method and device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can perform the steps in the above method.

[0200] The method provided in the embodiment of the present application can be applied to electronic devices with display functions. The electronic devices include terminal devices. The specific device form of the terminal device can refer to the above related descriptions and will not be repeated here.

[0201] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0202] For example, FIG9 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. As shown in FIG9, the terminal device includes: the terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0203] It should be understood that the structures illustrated in the embodiments of the present invention do not constitute specific limitations on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0204] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0205] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0206] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0207] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0208] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments of the present application, the terminal device may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0209] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via a wireless charging coil in the terminal device. While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141.

[0210] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0211] The terminal device implements display functionality through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0212] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the terminal device can include one or N display screens 194, where N is a positive integer greater than 1.

[0213] The terminal device can realize the shooting function through the ISP, camera 193, video codec, GPU, display 194 and application processor.

[0214] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0215] Camera 193 is used to capture still images or videos. In some embodiments, the terminal device may include one or N cameras 193, where N is a positive integer greater than 1. The types of the multiple cameras 193 can vary. For example, camera 193 may include a camera for capturing color images or a time-of-flight camera. In the embodiments of the present application, a camera can be understood as a camera. Cameras include: the AO camera, time-of-flight camera, etc. in the embodiments of the present application.

[0216] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0217] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area.

[0218] The terminal device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0219] Keys 190 include a power button, a volume button, and other buttons. Keys 190 can be mechanical or touch-sensitive. The terminal device can receive key inputs and generate key signal inputs related to user settings and function control of the terminal device. Indicator 192 can be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0220] In addition, on top of the above components, the device also runs an operating system, such as the iOS operating system, the Android operating system, or the Windows operating system. Applications can be installed and run on the operating system.

[0221] The software system of the terminal device can adopt a layered architecture, event-driven architecture, micro-kernel architecture, microservice architecture, or cloud architecture, etc., which will not be described here.

[0222] The present embodiment provides a chip. The chip includes a processor configured to invoke a computer program stored in a memory to execute the technical solution of the above embodiment. The implementation principles and technical effects are similar to those of the above-mentioned related embodiments and will not be further described here.

[0223] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned method is implemented. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0224] In one possible implementation, computer-readable media may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium designed to carry or store the desired program code in the form of instructions or data structures and accessible by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave are used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and optical disk as used herein include optical disk, laser disk, optical disk, digital versatile disk (DVD), floppy disk and Blu-ray disk, where disks generally reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0225] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0226] The present application embodiment is described with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processing unit of general-purpose computer, special-purpose computer, embedded processing machine or other programmable device to produce a machine, so that the instruction executed by the processing unit of computer or other programmable data processing device produces the device for realizing the function specified in one flow chart flow or multiple flows and / or one block or multiple blocks of block diagram.

[0227] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0228] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An interface display method, characterized in that: Applied to a terminal device, the terminal device comprises: a display screen and a display driver; the method comprises: When receiving the image transmitted by the display driver, the display screen refreshes and displays the image transmitted by the display driver; Wherein, when the display screen receives the Nth frame image transmitted by the display driver, it refreshes and displays the Nth frame image, and the frame rate corresponding to the drawing and rendering of the Nth frame image is the first frame rate; During a first duration of refreshing and displaying the Nth frame image, if the display screen does not receive the N+1th frame image transmitted by the display driver, the display screen does not read the Nth frame image from the cache and keeps displaying the Nth frame image; the first duration is greater than the cycle duration corresponding to the first frame rate; After a first period of time after the display screen refreshes and displays the Nth frame image, when the display screen receives the N+1th frame image transmitted by the display driver, the display screen refreshes and displays the N+1th frame image, and the corresponding frame rate when the N+1th frame image is drawn and rendered is a second frame rate, and the second frame rate is less than the first frame rate.

2. The method according to claim 1, characterized in that Before the display screen refreshes the display when receiving the image transmitted by the display driver, the method further includes: The terminal device obtains display screen brightness and / or ambient light intensity; The display screen refreshes the display when receiving the image transmitted by the display driver, including: When the brightness of the display screen is greater than a first value, and / or the ambient light intensity is greater than a second value, the display screen refreshes the display when receiving the image transmitted by the display driver.

3. The method according to claim 2, characterized in that The method further comprises: The terminal device determines a first refresh rate based on the brightness of the display screen and / or the ambient light intensity; In a case where the display screen does not receive the N+1th frame image transmitted by the display driver within a third time period after refreshing and displaying the Nth frame image, the display screen reads the Nth frame image from the cache and refreshes and displays the Nth frame image within a third time period after refreshing and displaying the Nth frame image, wherein the third time period is the cycle time period corresponding to the first refresh rate.

4. The method according to claim 3, characterized in that The terminal device stores a first corresponding relationship; the first corresponding relationship is the relationship between the brightness of the display screen and the first refresh rate; The terminal device confirms the first refresh rate based on the brightness of the display screen and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the brightness of the display screen and the first corresponding relationship; Alternatively, the terminal device stores a second corresponding relationship; the second corresponding relationship is the relationship between the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display brightness and / or the ambient light intensity, including: the terminal device confirms the first refresh rate based on the ambient light intensity and the second corresponding relationship; Alternatively, the terminal device stores a third corresponding relationship; the third corresponding relationship is the relationship between the display screen brightness, the ambient light intensity and the first refresh rate; the terminal device confirms the first refresh rate based on the display screen brightness and the ambient light intensity, including: the terminal device confirms the first refresh rate based on the display screen brightness, the ambient light intensity and the third corresponding relationship.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the display brightness is less than or equal to the first value, and the ambient light intensity is less than or equal to the second value When the display screen is refreshed at a preset refresh rate; the preset refresh rate is related to the foreground running application of the terminal device; Wherein, when the display screen receives the Mth frame image transmitted by the display driver, it refreshes and displays the Mth frame image, and the frame rate corresponding to the drawing and rendering of the Mth frame image is the third frame rate; In a case where the display screen does not receive the M+1th frame image transmitted by the display driver within a second time duration after refreshing and displaying the Mth frame image, the display screen reads the Mth frame image from the cache and refreshes and displays the Mth frame image during a second time duration after refreshing and displaying the first application interface, and the second time duration is equal to the cycle duration corresponding to the third frame rate; After receiving the M+1th frame image transmitted by the display driver, the display screen refreshes and displays the M+1th frame image. The frame rate corresponding to the drawing and rendering of the Nth frame image is the fourth frame rate, and the fourth frame rate is less than the third frame rate.

6. The method according to claims 1-5, characterized in that: The terminal device pre-stores an application list; The application corresponding to the Nth frame image is an application in the application list, and the application corresponding to the N+1th frame image is the same as the application corresponding to the Nth frame image.

7. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image including the bullet comments, and the N+1th frame image is an image not including the bullet comments.

8. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image corresponding to a first scene; the N+1th frame image is an image corresponding to a second scene, the first scene is a battle scene of a game application, and the second scene is a non-battle scene of the game application.

9. The method according to any one of claims 1 to 6, characterized in that: The Nth frame image is an image including motion effects, and the N+1th frame image is an image not including motion effects.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 9.

Images