Display screen refreshing method and electronic device

Abstract

The present application provides a display screen refreshing method and an electronic device. The method comprises: an electronic device calculates a parameter of an ROI on the basis of a content change region in a layer, performs synthesis to obtain image data of the ROI, and sends the parameter and image data of the ROI to a display driver. In a scenario in which the display driver continuously receives ROIs, the display driver not only controls, on the basis of the parameters of the ROIs, a corresponding local region in a display screen to refresh image data of each ROI, but also uses the lowest refresh frequency to periodically calculate a non-ROI in each cycle, and then drives a region corresponding to the non-ROI in the display screen to be refreshed. The non-ROI is calculated on the basis of a region other than each ROI within a current cycle. In a scenario in which the display driver non-continuously receives ROIs, the display driver uses the lowest refresh frequency to drive the display screen to refresh all regions. In this way, the refresh power consumption of the display screen is saved, the changed content is refreshed and displayed in real time, and the smoothness of a display effect is guaranteed.

Description

A display refresh method and electronic device Technical Field

[0001] This application relates to the field of terminals, and more particularly to a display refresh method and an electronic device. Background Technology

[0002] Currently, electronic devices are using increasingly higher refresh rates for their displays in order to provide better display effects. However, as high refresh rates are applied to displays, the associated power consumption problem is becoming more and more serious.

[0003] How to ensure the display effect of electronic devices while reducing their power consumption is an urgent problem to be solved.

[0004] Summary of the Invention

[0005] This application provides a display refresh method and an electronic device. The method includes: a layer compositing system of the electronic device acquiring layers provided by an application, calculating the parameters (i.e., location information) of the Region of Interest (ROI) based on the content-changing areas in the layers, and compositing the image data of the ROI, and sending the ROI parameters and image data to a display driver. In scenarios where ROIs are continuously received, the display driver not only controls the corresponding local area on the display to refresh the image data of each ROI based on the ROI parameters, but also periodically calculates the non-ROIs in each cycle using the lowest refresh rate, and then drives the area corresponding to the non-ROIs on the display to refresh. The non-ROIs are calculated based on areas outside each ROI in the current cycle. Furthermore, in scenarios where the display driver does not continuously receive ROIs, the display driver also drives the display to refresh all areas of the display using the lowest refresh rate. This saves display refresh power consumption and enables real-time refresh of the displayed content after changes, ensuring smooth display performance.

[0006] In a first aspect, this application provides a display screen refresh method applied to an electronic device, the electronic device including a display screen, the display screen being an OLED display screen or an LCD display screen, the method comprising: acquiring image data of a first layer at a first moment; acquiring image data of a second layer at a second moment, wherein the second moment is after the first moment; determining a first region of interest (ROI) in the second layer based on the image data of the first layer and the image data of the second layer, wherein the first ROI includes a first region, the first region being a region in the second layer where the image data of the second layer has changed relative to the image data of the first layer; sending the image data of the first ROI to the display screen; sending location information of the first ROI to the display screen, wherein the location information of the first ROI indicates a second region; and refreshing the image data of the first ROI in the second region of the display screen.

[0007] In conjunction with the method described in the first aspect, when the display screen refreshes the image data of the first ROI in the second region, the method further includes: the display screen does not refresh in areas other than the second region.

[0008] Implementing the method provided in the first aspect provides a partial refresh scheme for the display screen, reducing the power consumption of electronic devices. Specifically, after determining that the content change area of ​​the user interface is a partial area, only the corresponding partial area on the display screen is refreshed to display the changed content, without refreshing the rest of the display screen, thus reducing redundant refreshes and saving power consumption of electronic devices.

[0009] In conjunction with the method described in the first aspect, before sending the image data of the first ROI to the display screen, the method further includes: obtaining the image data of the first ROI by performing layer compositing based on the image data of the first ROI in the first layer.

[0010] In this way, during the layer compositing stage, it is not necessary to composite the image data of all layers; only the image data of the ROI in the layer needs to be composited, thereby further reducing the power consumption of the display refresh.

[0011] In conjunction with the method described in the first aspect, the first ROI is specifically the region obtained by extending the first region laterally to the column edge in the second layer.

[0012] Thus, when some electronic devices have displays that only support row-selective driving and not column-selective driving, extending the ROI area to the entire row can adapt to subsequent refresh modes that only support row-selective driving, thereby improving the feasibility of this solution.

[0013] In conjunction with the method described in the first aspect, the time interval between the first moment and the second moment is less than a preset time interval. After acquiring the image data of the second layer, the method further includes: sequentially acquiring the N ROIs corresponding to the N layers respectively, and through the display screen, based on the image data of the N ROIs corresponding to the N layers respectively, sequentially refreshing the areas indicated by the position information of the N ROIs on the display screen.

[0014] In this way, in scenarios with continuous image delivery, the ROI in each layer can be calculated and each ROI can be refreshed sequentially.

[0015] In conjunction with the method described in the first aspect, the display screen uses a first frequency to refresh the image data of the first ROI in the second region, and sequentially refreshes the image data of the N ROIs in the regions indicated by the position information of the N ROIs respectively; wherein the first frequency is greater than or equal to the second frequency, and less than or equal to the third frequency, the second frequency is the lowest refresh frequency supported by the electronic device, and the third frequency is the highest refresh frequency supported by the electronic device.

[0016] In this way, the display uses an adaptive refresh method to refresh the ROI image data. It can automatically reduce the refresh rate when the application sends fewer images, thereby reducing energy consumption, or automatically increase the refresh rate when the application sends more images, thus flexibly and intelligently adjusting the refresh rate.

[0017] In conjunction with the method described in the first aspect, after sequentially acquiring the N ROIs corresponding to the N layers respectively, the method further includes: acquiring non-ROIs, wherein the non-ROI is a continuous region in the Nth layer including a third region, wherein the third region is the region in the Nth layer other than the first ROI and the N ROIs; the display screen refreshes the image data of the non-ROI in a fourth region, wherein the fourth region is the region indicated by the location information of the non-ROI.

[0018] This can prevent abnormal issues such as leakage current or screen flickering from occurring in pixels in localized areas that have not been refreshed for a long time.

[0019] In conjunction with the method described in the first aspect, the method specifically includes: obtaining a second frequency corresponding to the time interval between obtaining the first ROI and obtaining the Nth ROI, wherein the second frequency is the lowest refresh frequency supported by the electronic device.

[0020] This allows for the use of the lowest refresh rate to refresh non-ROI image data, reducing power consumption while maintaining full-screen display quality.

[0021] In conjunction with the method described in the first aspect, before acquiring the non-ROI, the method further includes: among the acquired first ROI and the N ROIs, the interval between any two adjacent ROIs is less than the first time duration.

[0022] In this way, in continuous image delivery scenarios, the lowest refresh rate can be used to refresh non-ROI image data.

[0023] In conjunction with the method described in the first aspect, before acquiring the first ROI in the second layer, the method further includes: acquiring the second ROI in the first layer; sending image data of the second ROI to the display screen; sending location information of the second ROI to the display screen, wherein the location information of the second ROI indicates a fifth region; the display screen refreshing the image data of the second ROI in the fifth region; and within a first time period after acquiring the second ROI in the first layer, if it is determined that no ROI in other layers has been acquired, then the display screen refreshes the image data of the first layer in all regions.

[0024] In conjunction with the method described in the first aspect, after the display screen refreshes the image data of the first layer in all areas, the method further includes: based on the acquired first ROI, stopping the display screen from refreshing the image data of the first layer in all areas.

[0025] In this way, if no new ROI image data is acquired for a long time (exceeding the first time period), the entire area of ​​the display screen can be refreshed, thereby ensuring that the entire area that has not been refreshed for a long time can be refreshed at the lowest refresh rate, avoiding abnormal problems such as leakage or screen flickering of all pixels.

[0026] In conjunction with the method described in the first aspect, the time interval between the first moment and the second moment is longer than a preset time interval.

[0027] This ensures that the entire area is refreshed even in scenarios where no images have been sent for an extended period.

[0028] In conjunction with the method described in the first aspect, the display screen refreshes the image data of the first layer in all areas, specifically including: the display screen uses a second frequency to refresh the image data of the first layer in all areas, the second frequency being the lowest refresh frequency supported by the electronic device.

[0029] In this way, by periodically refreshing the entire area of ​​the display screen at the lowest refresh rate, it is possible to prevent the display from malfunctioning due to leakage current caused by some areas of the display screen not being refreshed for a long time during the partial refresh process. This ensures the normal display effect of the display screen and improves the user's visual experience.

[0030] In conjunction with the method described in the first aspect, the display screen includes a display driver integrated circuit (DDIC), a random access memory (DDIC RAM) of the DDIC, a display panel, sending image data of the first region of interest (ROI) to the DDIC RAM of the display screen; sending position information of the first ROI to the DDIC of the display screen; obtaining image data of the first ROI from the DDIC RAM based on the position information of the first ROI through the DDIC; and refreshing the image data of the first ROI in the second area of ​​the display panel.

[0031] In this way, the DDIC in the display can obtain the corresponding ROI image data from the DDIC RAM based on the ROI location information, thereby enabling only the area corresponding to the ROI in the display panel to be refreshed. This ensures accurate refreshing of the areas where content changes, avoids refreshing redundant areas, and greatly reduces the refresh power consumption of electronic devices.

[0032] In conjunction with the method described in the first aspect, the electronic device also includes a hardware synthesizer (HWC), which first acquires image data of a first layer and then acquires image data of a second layer; and uses the HWC to determine a first ROI in the second layer based on the image data of the first layer and the image data of the second layer.

[0033] In this way, HWC can calculate ROI based on layer information. Optionally, ROI can also be calculated based on layer information through SF in the framework layer. This application does not limit this.

[0034] In conjunction with the method described in the first aspect, the electronic device further includes a display driver; sending image data of the first ROI to the display screen, specifically including: sending the image data of the first ROI to the display driver via the HWC, and sending the image data of the first ROI to the display screen via the display driver; sending location information of the first ROI to the display screen, specifically including: sending the location information of the first ROI to the display driver via the HWC, and sending the location information of the first ROI to the display screen via the display driver.

[0035] In this way, HWC can send the ROI location information to the display screen through the display driver, thereby ensuring that the display screen can only refresh the local area indicated by the ROI location information.

[0036] In conjunction with the method described in the first aspect, the electronic device further includes a display synthesis system SF, a first module, an HWC, and a display driver. The display screen uses a first frequency to refresh the image data of the first ROI in the second region, and before sequentially refreshing the image data of the N ROIs in the regions indicated by the position information of the N ROIs, the method further includes: sending the third frequency to the HWC through the SF, sending the third frequency to the display driver through the HWC, and sending the third frequency to the display screen through the display driver; sending the second frequency to the display driver through the first module, and sending the second frequency to the display screen through the display driver.

[0037] This allows the minimum and maximum refresh rates to be sent to the display screen, ensuring that the screen can both adapt to the refresh rate and maintain the minimum refresh rate, thus avoiding display abnormalities.

[0038] In conjunction with the method described in the first aspect, the electronic device further includes a display driver. If, within a first duration after acquiring the second ROI in the first layer, it is determined that no ROI in other layers has been acquired, the display screen refreshes the image data of the first layer across the entire area. Specifically, this includes: starting a timer based on the acquired image data of the second ROI using the display driver; determining whether the timer's duration has reached the first duration using the display driver; if the timer's duration has reached the first duration, controlling the display screen to refresh the entire area of ​​the display screen based on the image data of the first layer using the display driver; and restarting the timer based on the acquired image data of a new ROI using the display driver.

[0039] In this way, the interval of each ROI can be calculated through the display driver. When a new ROI is received, the interval of each ROI is recalculated. When the frame interval exceeds the first duration, it can be indicated that the non-continuous image delivery scenario has been entered. When the frame interval does not exceed the first duration, it can be indicated that the continuous image delivery scenario has been entered. This enables the refresh strategies corresponding to the non-continuous and continuous image delivery scenarios.

[0040] In conjunction with the method described in the first aspect, the display screen refreshes the image data of the first layer in the entire area, specifically including: controlling the display screen through the display driver, using a second frequency, and refreshing the entire area of ​​the display screen based on the image data of the first layer; the second frequency is the lowest frequency supported by the electronic device.

[0041] In a second aspect, this application provides an electronic device including one or more memories, one or more processors, and a computer program stored on the memory, a display screen, the processor executing the computer program to implement the steps described in any of the first aspects.

[0042] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps described in any of the first aspects.

[0043] Fourthly, this application provides a computer program product including a computer program / instructions that, when executed by a processor, implement the steps described in any of the first aspects. Attached Figure Description

[0044] Figures 1A-1C illustrate a set of content change scenarios for a local area in a user interface provided in an embodiment of this application.

[0045] Figures 2A-2C are schematic diagrams of a set of scenarios for determining ROI based on content change areas provided in the embodiments of this application;

[0046] Figure 3 shows several refresh frequencies involved in the display refresh mechanism provided in the embodiments of this application;

[0047] Figures 4A-4C illustrate the display refresh mechanisms for non-continuous image feeding scenarios and continuous image feeding scenarios provided in the embodiments of this application.

[0048] Figure 5 is a flowchart of the display screen refresh method provided in an embodiment of this application;

[0049] Figure 6 illustrates the OS interaction involved in the display refresh method provided in this application embodiment;

[0050] Figure 7 is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application;

[0051] Figure 8 is a schematic diagram of the software architecture of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0053] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0054] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0055] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with a user. It realizes the conversion between the internal form of information and the form that the user can receive. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0056] First, let me introduce the relevant terminology used in this application.

[0057] Layer Compositing System: The layer compositing system comprises the Surface Flinger (SF) and the Hardware Composer (HWC). Both SF and HWC run on the application processor (AP) side of the electronic device. SF, as a module in the framework layer, is responsible for handling upper-layer compositing tasks, including retrieving layers from buffers of one or more applications (such as the primary application, system UI, etc.) and compositing image frames based on these layers. HWC, as a module in the hardware abstraction layer and an auxiliary component of SurfaceFlinger, is responsible for handling lower-level compositing tasks, including receiving layer information to be displayed from SF and optimizing the layer compositing process through hardware acceleration.

[0058] In the embodiments of this application, the layer compositing system is also used to determine the ROI. The definition of ROI and the method for determining ROI can be found in the detailed description below, and will not be repeated here.

[0059] Advanced Graphics Platform (AGP) is a graphics service developed by electronic device manufacturers. It runs on the AP side of the electronic device and can be used to control the refresh rate of the display screen.

[0060] In this embodiment, AGP can set a matching minimum refresh rate (Minimum frames per second, minfps) and maximum refresh rate (Max frames per second, maxfps) based on the current application scenario of the electronic device. The minfps is sent to the DDIC via Hwdisplay, display driver, etc., so that the DDIC performs a global refresh according to the minfps, or refreshes non-ROIs within the current cycle at the end of each cycle according to the minfps. The maxfps is also sent to the DDIC via SF, HWC, display driver, etc., enabling the DDIC to implement adaptive refresh technology within the range of maxfps and minfps. Optionally, when applying image data rendering, minfps and maxfps refer to the frequency of applying image data rendering, with the unit being fps. When the display refreshes image data, minfps and maxfps refer to the frequency of the display refreshing image data, with the unit being Hz. For details, please refer to the following description; further elaboration is omitted here.

[0061] hwdisplay is a service in HAL. In this embodiment, the hwdisplay service provides a corresponding interface to receive the refresh rate sent by AGP, such as minfps, and sends the minfps to DDIC through the display driver.

[0062] The display screen includes a display driver integrated circuit (DDIC), a DDIC random access memory (RAM), and a display panel. The DDIC can read image data to be displayed from the DDIC RAM and convert the image data into electrical signals to control the panel to refresh and display the image data. The DDIC RAM stores image data written by the application processing unit (AP), which is the image data synthesized by a layer compositing system running on the AP side. The panel is responsible for displaying the corresponding user interface under the control of the DDIC. The panel can be a liquid crystal display (LCD) or an organic light-emitting diode (OLED), etc. In this embodiment, the display screen can be a display screen independent of the electronic device or a display screen integrated into the electronic device; this embodiment does not impose any limitations on this.

[0063] In this embodiment of the application, DDIC can control the refresh of the corresponding local area in the Panel to display the image data of the ROI based on the image data of the ROI in DDIC RAM. The specific implementation method of the local refresh can be referred to in the detailed description below, and will not be repeated here.

[0064] Region of Interest (ROI): ROI refers to the region calculated based on the content-changing areas within a layer during the layer compositing stage of the layer compositing system, which combines layers into image data. In this embodiment, the ROI includes the content-changing areas, or the ROI is the same as the content-changing areas. The calculation method for ROI can be found in the following descriptions of Figures 2A-2C, and will not be elaborated upon here.

[0065] In this embodiment, the synthesized image data and the display area of ​​the screen have a fixed mapping relationship. Therefore, after the layer compositing system determines the Region of Interest (ROI) in the layer, the display driver can determine the area corresponding to the ROI on the screen and the area corresponding to the displayed user interface, etc., based on the mapping relationship. In other words, the ROI can refer to an area in the synthesized image data, an area on the screen, or an area in the user interface, etc.

[0066] To illustrate with a specific example, when the video playback window in the user interface to be displayed changes, the ROI includes the video playback window, as detailed in Figures 1A and 2A below; when the text input window in the user interface to be displayed changes, the ROI includes the text input window, as detailed in Figures 1B and 2B below; and when a foldable screen is used, when the content of the sub-screen in the user interface to be displayed changes, the ROI includes the content of the sub-screen, as detailed in Figures 1C and 2C below.

[0067] Global refresh: refers to refreshing the entire area (i.e., all pixels) of the Panel when DDIC controls the Panel to refresh the user interface.

[0068] Partial refresh: This refers to the process of refreshing a specific area (i.e., a portion of pixels) of the Panel when DDIC controls the Panel's user interface refresh, while the rest of the Panel remains unrefreshed. The local area of ​​the Panel corresponds to the Region of Interest (ROI) calculated by the layer compositing system. The image data refreshed and displayed in the local area is the ROI's image data, representing the changed content within the layer. The remaining areas of the Panel, excluding the local area, are used to display non-ROI image data, which also represents the unchanged content within the layer.

[0069] Refresh rate (also known as frame rate): At the software level, it refers to the number of frames displayed per unit of time; at the hardware level, it refers to the number of times the image data on the panel is refreshed per unit of time. The unit of refresh rate can be the number of times per second (Hertz, Hz). The higher the refresh rate of the display, the smoother the display effect, but correspondingly, it will consume more power. Common refresh rates for electronic devices include, but are not limited to: 60Hz, 75Hz, 90Hz, 120Hz, and 144Hz.

[0070] In one display refresh technology, an e-ink screen can be used to achieve partial refresh. Specifically, this includes: sending layer data drawn based on touch-press operations to a hardware abstraction layer for synthesis to obtain a first synthesized image; determining a first change area based on the first and second synthesized images; if the first change area is located within a preset range near the touch-press operation, then determining that the e-ink screen is in handwriting mode; after determining that the e-ink screen is in handwriting mode, obtaining each synthesized image from the hardware abstraction layer for preprocessing; and displaying the preprocessed synthesized images in parallel through the e-ink screen.

[0071] Although e-ink displays can achieve partial refresh rates to improve writing responsiveness, their refresh rates are limited by their hardware driving principles and cannot reach those of LCD and OLED displays. Specifically, e-ink displays are composed of numerous microcapsules containing pigment particles with different charges. E-ink displays use electrophoresis technology to form images by the floating and undulating of charged pigment particles in the microcapsules under the influence of an electric field. Because the movement of pigment particles under the influence of an electric field is slower than the rearrangement of liquid crystal molecules, the refresh rates supported by e-ink displays are much lower than those of LCD and OLED displays.

[0072] Furthermore, in the aforementioned partial refresh technology for e-ink screens, the entire layer is synthesized first, then a local area within the layer is determined, and finally that local area is refreshed without refreshing the rest of the entire layer. This results in the problem of synthesizing redundant data and wasting power. Moreover, e-ink screens have a low refresh rate, and the corresponding refresh power consumption is much lower than that of LCD and OLED displays. Therefore, applying partial refresh in e-ink screens is not the optimal solution for reducing power consumption.

[0073] Secondly, the aforementioned partial refresh technology for e-ink screens can only achieve partial refresh for handwriting scenarios, and cannot cover all application scenarios. It does not provide comprehensive partial refresh technology, nor does it fundamentally provide a feasible partial refresh solution.

[0074] In summary, because the driving principles of e-ink screens differ significantly from those of LCD and OLED displays, e-ink screens do not support adaptive refresh rates and do not suffer from issues such as leakage or screen flickering (once an image is refreshed on an e-ink screen, it will not disappear even if the power is turned off). Therefore, achieving partial refresh using LCD and OLED displays remains an urgent problem to be solved.

[0075] In another display refresh technology, the DDIC in the display can pre-set the local refresh enable state for each frame of local image. During the local refresh display process, the application processor (AP) in the display sends the local image of each frame to the DDIC. The DDIC refreshes the screen panel based on the local image of each frame, thus achieving local refresh display. In this process, when refreshing each frame of local image, the DDIC can first read the local refresh enable flag to determine the local refresh enable state corresponding to the current frame. Then, based on the local refresh enable state, it determines the target refresh strategy. If the local refresh enable state is off, the target refresh strategy includes global refresh display, and the DDIC can perform global refresh display of the local image. If the local refresh enable state is on, the target refresh strategy includes local refresh display, and the DDIC can perform local refresh display of the local image. In the above method, during the partial refresh display process of DDIC, the local image of each frame can be displayed either globally or locally, determined by the local refresh enable state for each frame. Therefore, multiple global refresh displays can occur during the partial refresh display process, thus achieving partial refresh timing nested within the global refresh timing logic to complete the partial refresh display. Furthermore, the AP will set DDIC to a non-partial refresh state after a certain period, that is, periodically disabling local refresh and adopting global refresh.

[0076] However, the aforementioned display refresh technologies only involve the DDIC performing partial or global refresh of a local image based on a partial refresh enable flag. They do not specify what constitutes a local image, or in what scenarios the DDIC enables partial refresh, or in what states it disables it. In other words, they do not disclose the scenarios for partial or global refresh of a local image. Furthermore, the AP's method of periodically disabling partial refresh requires frequent switching on and off of the partial refresh, which also generates power consumption.

[0077] Furthermore, in one practical application, the content of the user interface does not change completely before and after the refresh of the Panel display. It is possible that only the content of a local area changes before and after the refresh, while the content of the rest of the area remains unchanged. This depends on whether the content provided by the application that makes up the user interface changes.

[0078] It is evident that if the aforementioned display refresh technology still fails to solve the following problems: how to reduce the power consumption caused by display refresh while ensuring the display effect of the content to be displayed by the application, for example, how to refresh the changed content in a timely manner to provide users with a smooth, real-time visual experience, etc.

[0079] To address the aforementioned issues, this application provides a display refresh method and an electronic device. The method includes: a layer compositing system of the electronic device acquiring layers provided by an application, calculating the location information of Regions of Interest (ROIs) and synthesizing image data of the ROIs based on the content-changing areas within the layers, and sending the ROI location information and image data to a display driver. In scenarios where ROIs are continuously received, the display driver not only controls the corresponding local area on the display screen to refresh the image data of each ROI based on the ROI location information, but also periodically calculates non-ROIs in each cycle using the lowest refresh rate, and then drives the corresponding areas on the display screen to refresh. The non-ROIs are calculated based on areas outside each ROI in the current cycle. Furthermore, in scenarios where the display driver does not continuously receive ROIs, the display driver also drives the display screen to refresh all areas using the lowest refresh rate. This saves display refresh power consumption and enables real-time refresh of changed content, ensuring smooth display performance.

[0080] In a specific local refresh process, the DDIC in the display also uses refresh rate adaptive technology based on the activity level of the ROI image data to dynamically adjust the local refresh rate of the display within the range of the highest and lowest refresh rates. The activity level of the ROI image data can be measured, for example, by the time interval between the DDIC RAM receiving each frame of ROI image data sent by the display driver. For instance, the shorter the interval between the DDIC RAM receiving the ROI image data sent by the display driver, the higher the activity level of the ROI image data. Therefore, when using refresh rate adaptive technology, a higher refresh rate can be dynamically set to refresh the ROI image data.

[0081] Implementing the display refresh method provided in this application can bring the following beneficial effects:

[0082] (1) Provide a partial refresh scheme for the display screen to reduce the power consumption of electronic devices. Specifically, after determining that the content change area of ​​the user interface is a partial area, only the corresponding partial area of ​​the display screen is refreshed to refresh the changed content, without refreshing the rest of the display screen, thereby saving the power consumption of electronic devices.

[0083] (2) Ensuring full-screen display effect. Specifically, in scenarios where the display driver continuously receives ROIs (also known as continuous image delivery scenarios), the DDIC in the display screen, in addition to adaptively refreshing the refresh rate within the refresh rate range to control the local area of ​​the Panel to refresh the ROI image data, will also use the lowest refresh frequency to control the other local areas of the Panel to refresh the non-ROI image data. This ensures that during continuous local refresh, areas that have not been refreshed for a long time can be refreshed at the lowest refresh rate, avoiding abnormal problems such as pixel leakage or screen flickering in these areas. Furthermore, in scenarios where the display driver has not received ROIs for a long time (also known as non-continuous image delivery scenarios), the DDIC in the display screen can use the lowest refresh frequency to control the refresh of all areas in the Panel, ensuring that in scenarios where no images are delivered for a long time, all areas that have not been refreshed for a long time can be refreshed at the lowest refresh rate, avoiding abnormal problems such as pixel leakage or screen flickering in all areas.

[0084] (3) Eliminating the need for periodically switching the partial refresh function further optimizes the power consumption of electronic devices. Specifically, for some display refresh technologies that use a fixed period to disable partial refresh, frequently switching the partial refresh function on and off is not the optimal solution for power consumption. Therefore, the strategy of setting a minimum refresh rate adopted in this application can be based on the minimum refresh rate to statistically analyze the set of non-ROIs refreshed in the first n-1 frames of each period, and calculate the ROIs that need to be refreshed on the display in the nth frame. Then, the set of non-ROIs in the first n-1 frames and the ROIs in the nth frame are refreshed. In this way, there is no need to periodically disable the partial refresh function, and all pixels on the display can be guaranteed to meet the minimum refresh rate, without abnormal problems such as leakage or screen flickering due to long-term lack of refresh.

[0085] Next, we will introduce the relevant application scenarios involved in the display refresh method provided in this application.

[0086] Figures 1A-1C are schematic diagrams illustrating content change scenarios in a local area of ​​a user interface provided in this application.

[0087] Figure 1A illustrates an example of a scenario involving content changes in a local area when running a social application.

[0088] As shown in Figure 1A, user interface 11 is displayed before the screen of the electronic device refreshes, and user interface 12 is displayed after the screen refreshes. The layer corresponding to user interface 11 in Figure 1A can be called the first layer, and the layer corresponding to user interface 12 can be called the second layer. If N user interfaces are refreshed after user interface 12, these N user interfaces can each correspond to N layers. The moment when the electronic device acquires image data from the first layer can be called the first moment, and the moment when the electronic device acquires image data from the second layer can be called the second moment. The first moment is before the second moment, and when the interval between the first and second moments is less than a preset interval, it indicates that the application is in a continuous image delivery scenario. When the interval between the first and second moments is greater than or equal to the preset interval, it indicates that the application is in a non-continuous image delivery scenario. The preset interval can be the same as or similar to the first duration for whether the display driver statistical ROI frame interval is satisfied, as described later; this application embodiment does not limit this. User interface 11 includes a video playback window 111 provided by a social application, which displays a frame from a fitness tutorial video.

[0089] The user interface 12 includes a video playback window 121 provided by a social application. The video playback window 121 displays a frame after the frame in the aforementioned video playback window 111. The rest of the content in the user interface 12 is the same as the rest of the content in the user interface 11.

[0090] After implementing the display refresh method provided in this application, when the display refreshes from the display user interface 11 to the display user interface 12, it actually only refreshes the ROI area corresponding to the video playback window 121 displayed on the display, rather than refreshing the entire display. For details on this implementation, please refer to the description in the method flow section below, which will not be elaborated here.

[0091] Figure 1B illustrates an example of a scenario involving content changes in a local area when running a messaging application.

[0092] As shown in Figure 1B, user interface 13 is displayed before the electronic device's screen refreshes, and user interface 14 is displayed after the screen refreshes. The layer corresponding to user interface 13 in Figure 1B can be called the first layer, and the layer corresponding to user interface 14 can be called the second layer. If N user interfaces are refreshed after user interface 14, these N user interfaces can each correspond to N layers. The moment when the electronic device acquires image data from the first layer can be called the first moment, and the moment when the electronic device acquires image data from the second layer can be called the second moment. The first moment is before the second moment, and when the interval between the first and second moments is less than a preset interval, it indicates that the application is in a continuous image delivery scenario. When the interval between the first and second moments is greater than or equal to the preset interval, it indicates that the application is in a non-continuous image delivery scenario. The preset interval can be the same as or similar to the first duration for determining whether the display driver statistical ROI frame interval is satisfied, as described later; this embodiment does not impose any limitations on this.

[0093] The user interface 13 includes a text input window 131 provided by a messaging application. This text input window 131 includes a text input field and keyboard controls. Users can enter text into the text input field by clicking the corresponding keyboard controls. When a user clicks a keyboard control, the corresponding control will be selected, and the corresponding text will be displayed in the text input field.

[0094] The user interface 14 includes a text input window 141 provided by a messaging application, which includes a text input field and keyboard controls. The text input field displays the user-entered text "Hello, Ms. Li". The rest of the content in user interface 14 is identical to that in user interface 13.

[0095] After implementing the display refresh method provided in this application, when the display refreshes from the display user interface 13 to the display user interface 14, it actually only refreshes the ROI area corresponding to the text input window 141 displayed on the display, rather than refreshing the entire display. For details on this implementation, please refer to the description in the method flow section below, which will not be elaborated here.

[0096] Figure 1C illustrates an example of a local area content change scenario in a foldable screen setting.

[0097] As shown in Figure 1C, when the display screen of the electronic device is a foldable screen, user interface 15 is displayed before the display screen refreshes, and user interface 16 is displayed after the display screen refreshes. The layer corresponding to user interface 15 in Figure 1C can be called the first layer, and the layer corresponding to user interface 16 can be called the second layer. If N user interfaces are refreshed after user interface 16, these N user interfaces can each correspond to N layers. The moment when the electronic device acquires the image data of the first layer can be called the first moment, and the moment when the electronic device acquires the image data of the second layer can be called the second moment. The first moment is before the second moment, and when the interval between the first moment and the second moment is less than a preset interval, it can be characterized that the application is in a continuous image delivery scenario. When the interval between the first moment and the second moment is greater than or equal to the preset interval, it can be characterized that the application is in a non-continuous image delivery scenario. The preset interval can be the same as or similar to the first duration for whether the display driver statistical ROI frame interval is satisfied, as described later; this embodiment does not impose any limitations on this.

[0098] The user interface 15 includes an advertising page 151 on the left screen and a shopping application page 152 on the right screen. Page 151 displays a tooltip control 151A, which users can click / swipe to view advertising details. Page 152 displays recommended product information, and the shopping application can refresh the product information periodically.

[0099] The user interface 16 includes an advertising page 161 on the left screen and a shopping application page 162 on the right screen. Page 162 displays newly recommended product information from the shopping application after a refresh. The rest of the content in user interface 16 is identical to that in user interface 15.

[0100] After implementing the display refresh method provided in this application, when the display refreshes from the display user interface 15 to the display user interface 16, it actually only refreshes the ROI area corresponding to the page 162 provided by the shopping application, rather than refreshing the entire display. For details on the specific implementation, please refer to the description in the method flow section below, which will not be elaborated here.

[0101] As shown in Figures 1A-1C, during the process of refreshing the user interface on the screen, only a portion of the content changes compared to the original user interface; the rest remains unchanged. Typically, the changed content is provided by the upper-layer application, such as the aforementioned social / video / shopping applications.

[0102] It is understood that the partial content change scenarios shown in Figures 1A-1C are merely examples. The partial content change scenarios covered in this application may also include others, and there is no specific limitation on the number of partial areas.

[0103] Based on the application scenarios introduced above, the method for determining ROI involved in the display refresh method provided in this application will be introduced next.

[0104] Figures 2A-2C are a set of schematic diagrams illustrating the scenario of determining ROI based on content change areas provided in this application.

[0105] As shown in Figure 2A, in conjunction with the scenario shown in Figure 1A above, the modules involved in determining the ROI based on the content change area of ​​the electronic device include, but are not limited to, SF and HWC running on the AP side, and specifically involve the following processing stages.

[0106] (a) SF determines the visible layers. Specifically, SF can calculate the visible layers during the layer compositing process and then send the data containing the visible layers to HWC.

[0107] In this context, visible layers refer to the layers that are in focus, i.e., the layers contained in the user interface provided by the application running in the foreground. These layers are composited and displayed to the user, hence the name visible layers. Taking the scenario shown in Figure 1A as an example, the visible layers calculated by SF include the layers provided by the System UI, namely Layer1, and the layers provided by social applications, namely Layer2. Layer1 and Layer2 can constitute a frame. Each frame composed of Layer1 and Layer2 is composed of various Views arranged and combined. For example, Layer1 includes Views (not shown in Figure 2A) that provide the aforementioned cellular signal indicator, wireless network indicator, Bluetooth indicator, battery indicator, and time indicator, respectively. Another example is Layer2, which includes Views that provide the aforementioned video playback window 121.

[0108] (b) HWC determines the ROI based on the content change area in each frame layer. Specifically, after receiving the layer transmitted by SF, HWC can compare the latest received current frame layer with the previous frame layer to determine the content change area in the current frame layer, and then determine the corresponding ROI based on the content change area.

[0109] In this context, the content-changed area refers to the area where pixels change. Taking the scene shown in Figure 1A as an example, in a frame composed of Layer 1 and Layer 2, because the pixels of the video playback window 121 in the second layer have changed relative to the video playback window 111 in the first layer, the area where the View used to display the video playback window 121 is located is the content-changed area (also known as the first area). The pixels in the remaining areas have not changed.

[0110] The determination of ROI regions based on content change areas includes: treating the entire row containing each content change area as a ROI; in other words, extending the content change areas horizontally to the column edges of the layer to form the ROI. Taking the scenario shown in Figure 1A as an example, the entire row containing the View used to display the video playback window 121 is taken as the ROI (also known as the first ROI). That is, the ordinate of each vertex of the ROI is the same as the ordinate of each vertex in the View, while the abscissas of the two vertices to the left of the ROI are the minimum value (0), and the abscissas of the two vertices to the right of the ROI are the maximum value. This embodiment of the application does not impose specific restrictions on these maximum values.

[0111] In another implementation of ROI determination in this application, the ROI can be determined by the Native layer in the Framework layer, and then the ROI and image data can be sent to HWC. This application does not limit the specific implementation method for determining the ROI.

[0112] Based on the foregoing analysis, the ROI determined in this application is usually larger than the actual content change area. Only when the actual content change area is an entire row will the ROI area be exactly the same as the actual content change area. This application uses the entire row of the content change area as the ROI because most electronic devices currently have displays that require linear writing of new data to each row of content under the drive of a row scanning signal, and cannot select columns to write new data. In another optional embodiment of this application, when the display of the electronic device simultaneously indicates row-selective driving and column-selective driving, the ROI determined in this application can be exactly the same as the actual content change area.

[0113] (c) Optionally, if the aforementioned determined content change area is multiple areas, i.e., multiple ROIs are calculated, HWC can also combine multiple ROIs into one ROI. Herein, HWC combines multiple ROIs into one ROI by combining multiple ROIs and the interval areas of multiple ROIs into a final ROI, which will not be elaborated here.

[0114] As shown in Figure 2B, in conjunction with the scenario shown in Figure 1B above, the modules involved in determining the ROI based on the content change area of ​​the electronic device include, but are not limited to, SF and HWC running on the AP side, and specifically involve the following processing stages.

[0115] (a) SF determines the visible layer. The principle of SF determining the visible layer is the same in different scenarios. Therefore, for an introduction to this stage, please refer to the previous description of SF determining the visible layer in the scenario of Figure 1A. It will not be repeated here.

[0116] Taking the scenario shown in Figure 1B as an example, the visible layers calculated by SF include the layer provided by the System UI, namely Layer 3, and the layer provided by the SMS application, namely Layer 4. Layer 3 and Layer 4 can constitute a frame. The frame composed of Layer 3 and Layer 4 is composed of various Views arranged and combined. For example, Layer 3 includes Views (not shown in Figure 2B) for providing the aforementioned cellular signal indicator, wireless network indicator, Bluetooth indicator, battery indicator, and time indicator, respectively. Layer 4 includes Views for providing the aforementioned text input window 141.

[0117] (b) HWC determines the ROI based on the content change area in the layer. The principle of HWC determining the ROI based on the content change area in the layer is the same in different scenarios. Therefore, for the introduction of this stage in the scenario of Figure 1B, please refer to the previous description of HWC determining the ROI based on the content change area in the layer in the scenario of Figure 1A. It will not be repeated here.

[0118] Taking the scenario shown in Figure 1B as an example, the pixels of the text input window 141 in the second layer change relative to the text input window 131 in the first layer. Therefore, the area where the View used to display the text input window 141 is located is the content-changed area (also known as the first area). The pixels of the remaining areas do not change. Therefore, the entire row where the View used to display the text input window 141 is located is taken as the ROI (also known as the first ROI). That is, the ordinate of each vertex of the ROI is the same as the ordinate of each vertex in the View, while the abscissa of the two vertices to the left of the ROI is the minimum value, i.e., 0, and the abscissa of the two vertices to the right of the ROI is the maximum value. This embodiment of the application does not impose a specific limitation on this maximum value.

[0119] (c) Optionally, if the aforementioned determined content change area consists of multiple areas, i.e., multiple ROIs are calculated, HWC can also combine multiple ROIs into one ROI. The principle by which HWC combines multiple ROIs into one ROI is the same in different scenarios. Therefore, for the introduction of this stage in the scenario of Figure 1B, please refer to the previous description of HWC combining multiple ROIs into one ROI in the scenario of Figure 1A, which will not be repeated here.

[0120] As shown in Figure 2C, in conjunction with the scenario shown in Figure 1C above, the modules involved in determining the ROI based on the content change area of ​​the electronic device include, but are not limited to, SF and HWC running on the AP side, and specifically involve the following processing stages.

[0121] (a) SF determines the visible layer. The principle of SF determining the visible layer is the same in different scenarios. Therefore, for an introduction to this stage, please refer to the previous description of SF determining the visible layer in the scenario of Figure 1A. It will not be repeated here.

[0122] Taking the scenario shown in Figure 1C as an example, the visible layers calculated by SF include the layer provided by the advertising page, namely Layer 5, and the layer provided by the shopping application, namely Layer 6. Layer 5 and Layer 6 can form a frame. The frame formed by Layer 5 and Layer 6 is composed of various Views arranged and combined. The various Views are not shown in detail in Figure 2C.

[0123] (b) HWC determines the ROI based on the content change area in the layer. The principle of HWC determining the ROI based on the content change area in the layer is the same in different scenarios. Therefore, for the introduction of this stage in the scenario of Figure 1B, please refer to the previous description of HWC determining the ROI based on the content change area in the layer in the scenario of Figure 1A. It will not be repeated here.

[0124] Taking the scenario shown in Figure 1C as an example, the pixel dimensions of the aforementioned advertising page 161 relative to the aforementioned advertising page 151 remain unchanged, and the pixel dimensions of the shopping application page 162 relative to the shopping application page 152 also remain unchanged. Therefore, the area where the shopping application page 162 is located is the content change area (also known as the first area). Thus, the shopping application page 162 is designated as the ROI (also known as the first ROI), meaning that the x and y coordinates of each vertex of the ROI are the same as the x and y coordinates of each vertex in page 162.

[0125] (c) Optionally, if the aforementioned determined content change area consists of multiple areas, i.e., multiple ROIs are calculated, HWC can also combine multiple ROIs into one ROI. The principle by which HWC combines multiple ROIs into one ROI is the same in different scenarios. Therefore, for the introduction of this stage in the scenario of Figure 1B, please refer to the previous description of HWC combining multiple ROIs into one ROI in the scenario of Figure 1A, which will not be repeated here.

[0126] Based on the application scenarios and ROI determination methods introduced above, the display refresh mechanism provided in this application will be introduced next.

[0127] First, referring to Figure 3, we will introduce several refresh frequencies involved in the display refresh mechanism provided in this application.

[0128] As shown in Figure 3, the display refresh mechanism provided in this application involves a Vertical Synchronization (Vsync) signal. The Vsync signal is a key signal used to synchronize the display refresh rate and the GPU's image frame rate. The Vsync signal is a periodic signal, optionally with a frequency of, for example, 120Hz. The Vsync signal is generated by the DDIC in the display. Whenever a pulse (also known as a TE signal) of the Vsync signal arrives, the DDIC feeds back the TE signal to the display driver running on the GPU to synchronize the timing of image data refresh with the display driver. In other words, the display driver may only write new image data to the DDIC RAM each time it receives a TE signal fed back by the DDIC.

[0129] Referring again to Figure 3, the display refresh mechanism provided in this application also involves a minimum refresh rate (minfps, also known as the second frequency) and a maximum refresh rate (maxfps, also known as the third frequency). Optionally, minfps is, for example, 10Hz, and maxfps is, for example, 120Hz. Specifically, minfps and maxfps are received by the DDIC from the display driver and then written into the corresponding registers. In a continuous image transmission scenario, the DDIC can read minfps and maxfps from the corresponding registers, and then, within the range of minfps and maxfps, dynamically adjust the refresh rate adaptively according to the activity level of the received ROI image data to achieve ROI refresh, that is, refresh each frame of ROI image data onto the Panel. The activity level of the ROI image data can be measured, for example, by the time interval between the DDIC receiving each frame of ROI image data sent by the display driver. For example, the shorter the interval between the DDIC receiving the ROI image data sent by the display driver, the higher the activity level of the ROI image data. Therefore, when using the adaptive refresh rate technology, a higher refresh rate can be dynamically set to refresh the ROI image data. For example, the longer the interval between the DDIC receiving image data of the ROI sent by the display driver, the lower the activity level of the image data representing the ROI. Therefore, when using adaptive refresh rate technology, a lower refresh rate can be dynamically set to refresh the image data of the ROI. This can further reduce the power consumption of refreshing the ROI. In addition, in continuous image sending scenarios, the display driver will also periodically count the non-ROIs in the current cycle based on minfps, and then control the display to ensure a minimum refresh rate to periodically refresh the non-ROIs. See the description of refresh method 2 in Figure 4B below for details. In this way, while reducing power consumption, it can also avoid abnormal display problems such as leakage and screen flickering caused by some pixels in the panel not being refreshed for a long time.

[0130] Referring again to Figure 3, the display refresh mechanism provided in this application also involves a partial reset refresh rate. Optionally, the partial reset refresh rate is 10Hz. The DDIC can set the partial reset refresh rate based on minfps, for example, setting the partial reset refresh rate to be the same as minfps and writing it to the corresponding register. In non-continuous image transmission scenarios, the DDIC can read the partial reset refresh rate from the corresponding register area and then use the partial reset refresh rate to achieve global refresh. See the description of refresh method 1 in Figure 4A below for details. In this way, while reducing power consumption, it can also avoid abnormal display problems such as leakage and screen flickering caused by some pixels in the panel not being refreshed for a long time.

[0131] In summary, the TE signal has the highest frequency, and the highest refresh rate in the adaptive refresh rate adjustment range is usually equal to or lower than the frequency of the TE signal. The partial reset refresh rate is the lowest, and it can usually be consistent with or lower than the lowest refresh rate.

[0132] Next, the display refresh mechanism in non-continuous image feeding scenarios and continuous image feeding scenarios will be introduced with reference to Figures 4A-4C.

[0133] Figure 4A illustrates an example of entering a discontinuous image delivery scenario and performing periodic global refresh. A continuous image delivery scenario refers to a scenario where the display driver continuously receives ROIs, while a discontinuous image delivery scenario refers to a scenario where the display driver does not receive ROIs for a long period (e.g., exceeding a first time interval). Typically, in the stage where the AP synthesizes ROI image data based on layers and sends the ROI image data to the display screen through the display driver to achieve refresh, if the interval between synthesizing each frame of ROI image data is long, the frame interval for the display driver to receive ROIs will also be long. Therefore, this application can use the scenario where no ROI image data is received for a long period (e.g., exceeding a first time interval) to characterize the discontinuous image delivery scenario.

[0134] Specifically, when an application running on the AP side does not display any new images for an extended period (i.e., the AP does not draw new layers for a long time), the layer compositing system on the AP side will have no layers to participate in compositing because it has not received any layers from the application for a long time. Therefore, the display driver will not receive the composited image data for a long time and will not send the corresponding instructions to the display screen. Based on this, after the display driver receives the ROI1 image data sent by the layer compositing system, it can start a timer for a first duration. If the timer reaches the first duration and no new image data (such as ROI2 image data) is received, it indicates that the electronic device has entered a discontinuous image delivery scenario. Therefore, the display driver will control the display screen to perform periodic global refreshes based on the partial reset frequency, i.e., the minimum refresh rate, until the display driver receives new image data (such as ROI2 image data), at which point it will control the display screen to stop performing global refreshes based on the minimum refresh rate.

[0135] Optionally, within the first time period after the display driver receives the image data of ROI1, the electronic device is still in a continuous image transmission scenario. In this continuous image transmission scenario, the display screen will dynamically adjust the display screen refresh rate within the range of the highest and lowest refresh rates based on the activity level of the ROI image data using adaptive refresh rate technology. This refreshes the image data of each newly received ROI. For example, within the first time period after the display screen's DDIC receives the ROI1CMD sent by the display driver, since no new ROI image data has been received, the DDIC may use adaptive refresh technology to either not refresh or continue refreshing the image data of ROI1. This application embodiment does not limit this.

[0136] Figure 4B illustrates an exemplary diagram of entering a continuous image delivery scenario, combining adaptive ROI refresh with periodic non-ROI refresh. A continuous image delivery scenario refers to a scenario where the display driver continuously receives ROIs. Typically, in the stage where the AP synthesizes ROI image data based on layers and sends the ROI image data to the display screen through the display driver to achieve refresh, if the interval between synthesizing each frame of ROI image data is short, the frame interval for the display driver to receive ROIs is also short. Therefore, this application can use the scenario of receiving ROI image data at short intervals (e.g., not exceeding a first duration) to characterize the continuous image delivery scenario.

[0137] Specifically, when an application running on the AP side needs to continuously change the screen (i.e., the AP continuously draws new layers), the layer compositing system running on the AP side will continuously compose image data (such as image data of ROI1-ROI5) based on the new layers of the application. Therefore, the display driver can continuously receive image data such as ROI1-ROI5 and then continuously send instructions corresponding to the image data of ROI1-ROI5 to the display screen. Based on this, after the display driver receives a new frame of image data sent by the layer compositing system, it can start a timer for a first duration. If new image data is received before the timer reaches the first duration, it indicates that the electronic device is in a continuous image delivery scenario, and the first timer is reset every time a new frame of image data is received. Therefore, in the continuous image delivery scenario, the display driver will control the activity level of the display screen based on the image data of ROI, and use refresh rate adaptive technology to dynamically adjust the display screen refresh rate within the range of the highest and lowest refresh rates, refreshing each newly received ROI image data until the display driver has not received image data sent by the layer compositing system for more than the first duration, at which point it will control the display screen to perform a global refresh based on the frequency of partial reset.

[0138] Specifically, in continuous image delivery scenarios, each frame of image data received by the display driver is specifically ROI image data. Each ROI image data frame may only correspond to pixels in a portion of the Panel. Therefore, the collection of all ROI image data received within a certain period may not completely cover all pixels of the Panel. In other words, some pixels in the Panel may not have been refreshed during that period. To avoid display anomalies caused by pixels not being refreshed for extended periods, the frequency of the partial reset signal described earlier can be used to periodically count non-ROIs within a cycle, and then refresh those non-ROIs at the end of the cycle.

[0139] Optionally, in continuous image delivery scenarios, the method for calculating non-ROIs includes: calculating each non-ROI based on the area outside each ROI within the current cycle, and then merging each non-ROI into a single non-ROI. The method for merging each non-ROI into a single non-ROI includes using the area formed by the coordinates of the top-left corner and the bottom-right corner as the final merged area. In this merging method, if each non-ROI is two consecutive areas, the final merged area is the union of these multiple non-ROI areas; if there are gaps between non-ROIs, the final merged area is the union of the gaps between these multiple non-ROIs and non-ROIs. Taking the layer corresponding to ROI5 shown in Figure 4B as the Nth layer as an example, when the Nth layer is at the end of the cycle corresponding to the lowest refresh rate, the non-ROIs for this cycle are determined. These non-ROIs are obtained by merging the areas outside the ROIs (also known as the third area) in each layer within the cycle into a consecutive area.

[0140] Optionally, in continuous image delivery scenarios, when calculating the non-ROIs within a cycle at the end of that cycle, the last ROI frame of the cycle (also known as the Nth frame) may be received, such as ROI5 as shown in Figure 4B. In this case, the non-ROIs within the cycle can be combined with ROI5 before the display refresh is controlled uniformly. The methods for combining the non-ROIs and ROI5 include: using the area formed by the coordinates of the top-left corner and the bottom-right corner as the final combined area; for example, using the area formed by the coordinates of the top-left corner of ROI5 and the coordinates of the priority angle of the non-ROI as the final combined area. Under this method, if the non-ROI and ROI5 are two consecutive areas, the final combined area is the union of these two areas (see Figure 4B for details); if the non-ROI and ROI5 are two discontinuous, spaced areas, the final combined area is the union of these two areas and the spaced area (see Figure 4C for details).

[0141] Based on the preceding introduction of application scenarios, ROI determination principles, and ROI-based display refresh mechanisms, the following section describes the process of the display refresh method provided in this application.

[0142] As shown in Figure 5, the method includes the following steps:

[0143] S51, Apply drawing layers.

[0144] Specifically, when an application needs to update the user interface, it can continuously provide new interface elements, and then draw and render these elements to obtain layers. The drawn layers can be continuously written to the corresponding buffers.

[0145] Optionally, when the application does not need to update the user interface, the application may not provide new interface elements, nor perform drawing or rendering layers, and the buffer may not have any new layers written.

[0146] S52, HWC calculates the ROI based on the content change area in the visible layer and sends it to the display driver.

[0147] Specifically, SF can continuously read layers from the buffer, and then SF and HWC synthesize image data based on the layers. During the layer synthesis stage, HWC can first calculate the ROI of the current frame based on comparing the content change areas in the layers of the previous and next frames, and then synthesize image data based on the ROI, continuously sending the ROI parameters and image data of each frame to the display driver.

[0148] Optionally, if the application does not require updating the user interface, HWC cannot calculate the ROI based on the content change area, and therefore does not send the ROI parameters and image data to the display driver.

[0149] The method used by HWC to determine the ROI includes: HWC can first determine the area of ​​content change in the visible layer, and then take the entire row containing the content change area as the ROI. For details, please refer to the description of Figures 2A-2C above, which will not be repeated here. It is evident that using the display method provided in this application, it is not necessary to composite the entire layer before displaying; instead, the ROI is composited before displaying.

[0150] The parameters of the ROI include its location information within the complete image data frame, including its row and column ranges, such as a row range of 0-1079 and a column range of 500-1499. The image of the ROI includes pixel information for each pixel within the ROI.

[0151] S53, the display driver refreshes the timer and restarts the timing every time it receives a new ROI.

[0152] Specifically, each time the display driver receives the latest frame of ROI image data, it refreshes the timer and starts timing again. This timer is used to time the frame interval of ROI reception. That is, after receiving a frame of ROI, the timer is started, and when a new frame of ROI is received, the timer is refreshed to its initial state and timing restarts.

[0153] Optionally, the timer can be a countdown timer for a first duration. When the frame interval is within the first duration range, it indicates that the current scene is a continuous image transmission scene, i.e., the subsequent steps S54-2 to S56 are executed; when the frame interval reaches the first duration, it indicates that the current scene is a non-continuous image transmission scene, i.e., the subsequent step S54-1 is executed. This application provides different refresh mechanisms for different image transmission scenes, as detailed below.

[0154] S54-1, after the timer reaches the first duration, controls the partial reset signal based on the minimum refresh rate, and performs a global refresh of the Panel based on the partial reset signal.

[0155] Specifically, when the timer in the display driver reaches its first duration, it triggers the display driver to activate the partial reset signal based on the minimum refresh rate. This partial reset signal occurs at the same frequency as the minimum refresh rate. The partial reset signal can be used by DDIC to periodically control the Panel to perform a global refresh. For a more detailed explanation of the periodic global refresh mechanism, please refer to the description of Figure 4A above; it will not be repeated here.

[0156] S54-2, determine whether the partial reset signal is enabled.

[0157] Specifically, before the timer reaches its first duration, the display driver needs to determine whether the partial reset signal is enabled based on the latest received ROI. If no ROI has been received for a long time, the partial reset signal is enabled at this time; if a new ROI has been received, the partial reset signal is disabled at this time. If the partial reset signal is disabled, the subsequent step S56 is executed directly; if the partial reset signal is enabled, the subsequent steps S55-S56 are executed first.

[0158] S55, if the partial reset signal is enabled, then the partial reset signal is disabled.

[0159] Specifically, after the display driver determines that the partial reset signal is enabled, it needs to disable the partial reset signal based on the newly received ROI, that is, it will no longer control the Panel to perform a global refresh based on the partial reset signal through DDIC.

[0160] S56, after the partial reset signal is turned off, the corresponding CMD command is determined based on the ROI parameters, and the image data of the corresponding local area in the Panel is refreshed using adaptive refresh rate control. Furthermore, the lowest refresh rate is used to control the refresh of the local areas in the Panel corresponding to non-ROIs.

[0161] Specifically, after the partial reset signal is turned off, the display driver will determine the corresponding command (CMD) in the Panel based on the parameters of the ROI, and then send the CMD command to the DDIC in the display screen, as well as send the image data of the ROI to the DDIC RAM, so that the DDIC controls the local area in the Panel to refresh the image data of the ROI.

[0162] Specifically, the CMD command refers to the display command set (dcs) instructions sent from MIPI to DDIC. These dcs instructions can modify register values, thereby changing the DDIC's operating mode. The dcs instructions also include parameters that DDIC can process, corresponding to the ROI's parameters. For example, if the ROI's row range is 0-1079 and its column range is 500-1499, the row range can be converted to the value "39 01 00 40 00 00 05 2A 00 00 04 37" in register 2A of DDIC, and the column range can be converted to the value "39 01 00 40 00 00 05 2B 01 F4 05DB" in register 2B of DDIC. DDIC then reads the ROI's image data from DDIC RAM based on the converted dcs instructions and controls the corresponding local area in the Panel to refresh the ROI's image data.

[0163] Optionally, in a continuous image delivery scenario—that is, when the frame interval is less than the first duration and ROI image data is continuously received—DDIC will also employ adaptive refresh rate technology based on the activity level of the ROI image data. This involves dynamically adjusting the refresh rate within a range of the highest and lowest refresh rates to control the refresh of each frame of ROI image data in the corresponding local area of ​​the Panel. The activity level of the ROI image data can be measured by the time interval between the DDIC RAM receiving each frame of ROI image data sent by the display driver. For example, a shorter interval indicates higher activity of the ROI image data; therefore, adaptive refresh rate technology allows for dynamically setting a higher refresh rate to refresh the ROI image data.

[0164] Optionally, during partial refresh, to avoid display anomalies caused by pixels in certain areas of the Panel not being refreshed for extended periods, DDIC will periodically refresh non-ROIs based on the lowest refresh rate. These non-ROIs are obtained by the display driver from areas outside the ROI set within the statistical period. The parameters of the non-ROIs are then converted into the aforementioned DCS command and sent to DDIC, which controls the refresh of the corresponding areas within the Panel. The method for determining non-ROIs can be found in the descriptions of Figures 4B-4C above, and will not be repeated here.

[0165] Based on the above analysis, during the display refresh of the user interface, if only part of the content changes, the image data of the Region of Interest (ROI) is determined based on the area of ​​content change. Then, only the corresponding local area on the display is refreshed with the image data of that ROI, thus saving display refresh power consumption. Furthermore, during local refresh, to prevent display abnormalities caused by pixels in some areas of the Panel not being refreshed for extended periods, areas of the Panel that do not correspond to ROIs are periodically refreshed. And, if no content changes in the user interface, the entire Panel is refreshed based on the lowest refresh rate to prevent display abnormalities caused by pixels in the Panel not being refreshed for extended periods.

[0166] The OS interaction involved in the display refresh method provided in this application will be described next.

[0167] As shown in Figure 6, the interactive modules involved in the display refresh method provided in this application include the AP and the display screen.

[0168] The AP side includes the first application running in the APP layer, SF and AGP in the framework layer, HWC and hwdisplay in the HAL layer, and the display driver in the Kernel layer. The display screen includes DDIC, DDIC RAM, and Panel. For a description of each module on the AP side, please refer to the description of the software architecture of the electronic device in Figure 8 below; it will not be repeated here. For a description of DDIC, DDIC RAM, and Panel, please refer to the introduction of relevant terminology used in this application above; it will not be repeated here.

[0169] As shown in Figure 6, the display method includes the following steps:

[0170] Phase 1 (S61-S68): The display driver receives the ROI sent from the upper layer.

[0171] S61, SF determines the visible layer based on the data read from the Buffer.

[0172] Specifically, the application at the APP layer (e.g., the first application) can provide interface elements, which are then drawn and rendered to obtain layers, and written to the corresponding buffers. SF can read the layers from the buffers and then determine the visible and invisible layers. Visible layers can participate in subsequent compositing, while invisible layers do not. Layer compositing can be performed by any one or more of SF and HWC, and this embodiment does not limit this.

[0173] Optionally, besides including only visible layers in the composition, it is also possible to not distinguish between visible and invisible layers, that is, to include all layers in the composition. In this case, SF does not need to distinguish between visible and invisible layers. This application does not limit this, and will only describe the example of including visible layers in the composition for now.

[0174] S62, SF sends the visible layer to HWC.

[0175] Specifically, after the SF determines the visible layer, it can also send the visible layer to the HWC, so that the HWC can synthesize image data based on the visible layer, or the HWC can assist the SF in synthesizing image data based on the visible layer.

[0176] Optionally, during the layer compositing stage, layer compositing can be achieved using only SF (Synchronous Filter) without the need for HWC (Heated Layer Concentrator). In this case, SF does not need to send visible layers or all layers to HWC. This application does not impose specific limitations on this, and will temporarily use the use of HWC to achieve layer compositing as an example for description.

[0177] S63, HWC calculates ROI based on the area of ​​content change in the visible layer.

[0178] Specifically, after HWC receives the visible layer sent by SF, HWC can first calculate the ROI based on the content change area in the visible layer, and then synthesize image data based on the ROI.

[0179] HWC's method for determining the ROI includes: HWC can first determine the area of ​​content change in the visible layer, and then take the entire row containing the content change area as the ROI. For details, please refer to the description of Figures 2A-2C above, which will not be repeated here. It is evident that using the display method provided in this application, it is not necessary to composite the entire layer before displaying; instead, the ROI is composited before displaying.

[0180] S64, HWC sends ROI parameters and image data to the display driver.

[0181] Specifically, after HWC determines the ROI, it can send the ROI's parameters and image data to the display driver. The display driver then performs the subsequent display tasks. The definition of the ROI's parameters can be found in the previous descriptions and examples, and will not be repeated here.

[0182] In S65, SF sends the highest refresh rate to the display driver via HWC.

[0183] Specifically, SF also sends the maximum refresh rate (denoted as maxfps) to HWC, and then HWC sends maxfps to the display driver. When adaptive refresh rate technology is subsequently adopted, the maximum refresh rate is limited by maxfps to refresh the image data of ROI.

[0184] The maximum refresh rate can be a preset fixed value or dynamically adjusted according to the application scenario of the electronic device. For example, when the electronic device is in an interactive state, in order to provide the user with a responsive, continuous, and smooth interactive experience, the maxfps can be set higher, such as 120Hz; when the electronic device is in a non-interactive state, the maxfps can be set lower, such as 90Hz. This application does not limit the specific value of maxfps; it can depend on the display's capabilities and the dynamic adjustment strategy for the application scenario.

[0185] It is understood that this application does not restrict the execution order of S65, but depends on whether the application scenario of the electronic device changes to trigger the SF to issue an update after maxfps.

[0186] In S66, AGP sends the minimum refresh rate to the display driver via hwdisplay.

[0187] Specifically, AGP has a preset minimum refresh rate (denoted as minfps). AGP can send minfps to hwdisplay, and then hwdisplay sends minfps to the display driver. This is used to control the refresh of the non-ROI area on the display screen, thus avoiding display abnormalities caused by the non-ROI area not being refreshed for a long time.

[0188] The minimum refresh rate can be a preset fixed value or it can be dynamically adjusted according to the application scenario of the electronic device. This application does not limit the specific value of minfps; it can depend on the display's capabilities and the dynamic adjustment strategy for the application scenario.

[0189] It is understood that this application does not restrict the execution order of S66, but depends on whether the application scenario of the electronic device changes to trigger the AGP to issue an update after the minfps.

[0190] S67, the display driver sends the highest refresh rate and lowest refresh rate to the DDIC.

[0191] Specifically, after receiving maxfps and minfps, the display driver can send maxfps and minfps to DDIC separately or simultaneously. This allows DDIC to use adaptive refresh rate technology based on the range of maxfps and minfps, or allows DDIC to refresh the non-ROI corresponding area on the display screen based on minfps.

[0192] S68, the display driver refreshes the timer and restarts the timing every time it receives new ROI image data.

[0193] Specifically, the display driver can distinguish between continuous image delivery scenarios and non-continuous image delivery scenarios based on the frame interval between the received ROI image data. Then, it provides different display refresh mechanisms for different scenarios. Therefore, when the display driver receives new ROI image data, it refreshes the timer and restarts the timing. For details, please refer to the specific implementation described in S53 above, which will not be elaborated here.

[0194] Phase 2 (S70-S79): Entering the continuous image delivery scenario.

[0195] In S70, the display driver determines whether partial reset is enabled; if it is, partial reset is disabled.

[0196] For a detailed introduction to S70, please refer to the descriptions in sections S54-2 to S55 above, which will not be repeated here.

[0197] S71, the display driver writes the ROI image data to the DDIC RAM in the display screen.

[0198] Specifically, after the display driver receives the image data of the ROI of a new frame, it needs to send the image data of the ROI to the display screen to refresh it, so it needs to be written to the DDIC RAM in the display screen.

[0199] Specifically, the DPU running the display driver can write ROI image data to the DDIC RAM in the display screen via the Mobile Industry Processor Interface (MIPI). This image data can be represented as a MIPI waveform. Because the data written by the display driver to the DDIC RAM in the display screen is ROI image data, rather than the image data of a complete frame, the amount of ROI image data written by the display driver to the DDIC RAM is less than the amount of data of a complete frame written by the display driver to the DDIC RAM in traditional refresh technologies.

[0200] S72, the display driver sends the CMD command corresponding to the ROI parameter to the DDIC in the display screen.

[0201] Specifically, the display driver can convert the ROI-based parameters into corresponding CMD commands that DDIC can recognize, and then send these CMD commands to DDIC. For an introduction to the ROI parameters and their corresponding CMD commands, please refer to the description in section S56 above; it will not be repeated here.

[0202] S73, DDIC reads ROI image data from DDIC RAM using ROI-based CMD commands.

[0203] Specifically, DDIC can recognize the range of image data included in CMD commands and then read the corresponding ROI image data from DDIC RAM.

[0204] S74, DDIC uses adaptive refresh rate control based on the highest and lowest refresh rates to refresh the image data of the corresponding area of ​​the Panel for the ROI.

[0205] Specifically, DDIC reads the image data of the corresponding ROI from DDIC RAM and can control the corresponding local area (also known as the second area) in the Panel to refresh the image data of that ROI. Optionally, in continuous image delivery scenarios (i.e., scenarios where the display driver timing frame interval is less than the first duration), DDIC can use refresh rate adaptive technology based on the activity level of the ROI's image data to dynamically adjust the refresh rate within the range of the highest and lowest refresh rates, thereby controlling the corresponding local area in the Panel to refresh the image data of the ROI for each frame.

[0206] The activity level of the ROI image data can be measured by the time interval between each frame of ROI image data received by the DDIC RAM from the display driver. For example, the shorter the interval between the DDIC RAM receiving the ROI image data from the display driver, the higher the activity level of the ROI image data. Therefore, when using the refresh rate adaptive technology, a higher refresh rate can be dynamically set to refresh the ROI image data.

[0207] S75, the display driver periodically calculates the non-ROI for each cycle based on the lowest refresh rate.

[0208] Specifically, after S68, based on the latest received ROI image data, the display driver enters a continuous image delivery scenario. Therefore, to avoid display abnormalities caused by some pixels in the Panel not being refreshed for a long time, the display driver also needs to ensure that these pixels are refreshed periodically at the lowest refresh rate. Furthermore, to save refresh power consumption, the display driver needs to periodically count the non-ROIs within a cycle at the lowest refresh rate, so that only the local area (also known as the fourth region) of the Panel corresponding to the non-ROI is refreshed, instead of refreshing the entire area of ​​the Panel.

[0209] Regarding the specific implementation principle of the display driver periodically calculating the non-ROI in each cycle based on the lowest refresh rate, please refer to the description in Figures 4B-4C above, which will not be repeated here.

[0210] S76, the display driver writes non-ROI image data to the DDIC RAM in the display screen.

[0211] Specifically, at the end of a display driver cycle, the non-ROI image data collected during this cycle needs to be sent to the display screen for refresh, which requires writing to the DDIC RAM in the display screen.

[0212] S77, the display driver sends the CMD command corresponding to the non-ROI parameter to DDIC.

[0213] Specifically, the display driver can convert non-ROI parameters into corresponding CMD commands that DDIC can recognize, and then send the CMD commands to DDIC.

[0214] S78, DDIC reads non-ROI image data from DDIC RAM based on the non-ROI CMD command.

[0215] Specifically, DDIC can recognize the range of image data included in CMD commands and then read the corresponding non-ROI image data from DDIC RAM.

[0216] S79, DDIC controls the refresh of non-ROI image data in the corresponding area of ​​the Panel based on the lowest refresh rate.

[0217] Regarding the specific implementation principle of DDIC controlling the refresh of non-ROI image data in the corresponding area of ​​the Panel based on the lowest refresh rate, please refer to the description in Figures 4B-4C above, which will not be repeated here.

[0218] Phase 3 (S80-S81): Entering the non-continuous image delivery scenario.

[0219] S80, when the display driver timer reaches the first duration, activates the partial reset signal based on the lowest refresh rate.

[0220] Specifically, after S68, once the timer started by the display driver reaches its first duration, it triggers the display driver to activate a partial reset signal based on the minimum refresh rate. This partial reset signal occurs at the same frequency as the minimum refresh rate and can be used to trigger DDIC to perform a global refresh of the Panel.

[0221] S81, based on the partial reset signal control, instructs the display driver to notify DDIC to refresh the panel globally at the lowest refresh rate.

[0222] Based on the display refresh method described above, the hardware and software architecture of the electronic device involved in the display refresh method provided in this application will be introduced next.

[0223] Electronic devices can be equipped with Or other portable terminal devices with different operating systems, such as mobile phones, tablets, desktop computers, laptops, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices and / or smart city devices, etc.

[0224] Figure 7 shows a schematic diagram of the structure of the electronic device 100.

[0225] Electronic device 100 may include: processor 110, external memory interface 120, internal memory 123, universal serial bus (USB) interface 130, sensor module 180, and display screen 194, etc. The sensor module 180 may include pressure sensor 180A and touch sensor 180K, etc.

[0226] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0227] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, data processing unit (DPU), graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0228] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0229] In this embodiment of the application, the processor 110 is used to work with the display screen 194 and others to jointly implement the display screen refresh method provided in this application. For the specific implementation of the display screen refresh method, please refer to the previous description of Figures 5-6, which will not be repeated here.

[0230] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include a Universal Serial Bus (USB) interface, etc. The USB interface 130 is an interface compliant with the USB standard specification, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transfer between the electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0231] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0232] Internal memory 123 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0233] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0234] In this embodiment of the application, the electronic device 100 may store an implementation program of the display refresh method in the aforementioned memory, such as a method for determining the maximum refresh rate, the minimum refresh rate, the ROI, and a parameter passing method.

[0235] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0236] In this embodiment, the pressure sensor 180A can be used to detect touch and click operations on the application interface in order to control the application to refresh the interface.

[0237] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0238] In this embodiment, the touch sensor 180K is used to detect touch and click operations in the application interface in order to control the application to refresh the interface.

[0239] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0240] Display screen 194 is used to display images, videos, etc. Display screen 194 includes the DDIC, DDIC RAM, and Panel described above. The Panel can be a liquid crystal display (LCD). The display panel can also be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniled, microLEDs, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0241] In this embodiment, the electronic device can control the display screen 194 to display the application interface, specifically the user interface shown in Figures 1A-1C as described above.

[0242] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0243] Figure 8 is a software structure block diagram of an electronic device 100 according to an embodiment of this application.

[0244] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0245] The application layer can include a series of application packages.

[0246] As shown in Figure 8, the application package may include a first application, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, and other applications. The first application can provide elements of the application interface; for example, the applications described in Figures 1A-1C above.

[0247] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0248] As shown in Figure 8, the application framework layer may include a view system service, a window manager service, a surface flinger (SF), and AGP, etc.

[0249] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0250] The Windows Manager service is used to manage window programs. It can retrieve screen size, determine if a status bar is present, lock the screen, and capture screenshots, among other things.

[0251] The Surface Flinger (SF) system is primarily responsible for compositing and displaying the image. Specifically, SF can work with HWC to composite image data based on layers, and then send the image data to the display screen via the display driver to show the corresponding user interface.

[0252] AGP can set a matching minimum refresh rate (minfps) and maximum refresh rate (maxfps) based on the current application scenario of the electronic device. The minfps is then sent to the DDIC via Hwdisplay, the display driver, etc., allowing the DDIC to perform a global refresh according to this minfps, or to refresh non-ROIs within the current cycle at the end of each cycle according to this minfps. The maxfps is also sent to the DDIC via SF, HWC, the display driver, etc., enabling the DDIC to implement adaptive refresh technology within the maxfps and minfps range. For details, please refer to the previous method flow description; it will not be repeated here.

[0253] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0254] The Hardware Abstraction Layer (HAL) sits between the kernel layer and the application framework layer, playing a crucial bridging role. Specifically, the HAL includes the Hardware Composer (HWC) and hwdisplay. HWC exists as an independent module within the HAL, separating the upper-layer software system from the lower-layer hardware abstraction layer. They run in different processes and communicate via Binder. HWC calculates the Region of Interest (ROI) and sends the ROI parameters and image data to the display driver, enabling the display driver to control the screen refresh rate. The hwdisplay service provides interfaces to receive the refresh rate (e.g., minfps) sent by AGP and transmits this minfps to the DPU via the display driver, which then sends it to the DDIC.

[0255] The kernel layer contains at least a display driver. The display driver is the component in the software architecture responsible for managing and controlling the operation of the display screen, thereby displaying the user interface on the screen. The display driver typically includes the following key functions: data management, responsible for allocating and managing image data buffers used to store image data to be displayed on the screen; display control, the display driver is also responsible for controlling the physical parameters of the display, such as refresh rate, resolution, and color mode; synchronization and refresh, the display driver typically works in conjunction with a vertical synchronization (VSync) signal to ensure smooth rendering and display of graphics. The VSync signal is usually provided by the underlying hardware, but can also be generated by software to keep rendering, drawing, compositing, and the display refresh rate synchronized. Specifically, in the embodiments of this application, the display driver can be used to time frame intervals to determine whether a continuous image delivery scenario has been entered. If a continuous image delivery scenario has been entered, non-ROIs will be periodically counted based on the lowest refresh rate; if a non-continuous image delivery scenario has been entered, a partial reset signal will be activated based on the lowest refresh rate.

[0256] It should be understood that the steps in the above-described method embodiments provided in this application can be implemented by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0257] This application also provides an electronic device that may include a memory and a processor. The memory may be used to store a computer program; the processor may be used to invoke the computer program in the memory to cause the electronic device to perform the method in any of the above embodiments.

[0258] This application also provides a chip system including at least one processor for implementing the functions involved in the methods performed by the electronic device in any of the above embodiments.

[0259] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0260] The chip system can consist of chips or include chips and other discrete components.

[0261] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0262] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0263] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0264] This application also provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method executed by the electronic device in any of the above embodiments.

[0265] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the electronic device in any of the above embodiments.

[0266] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0267] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk).

[0268] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0269] In summary, the above description is merely an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made according to the disclosure of the present invention should be included within the scope of protection of the present invention.

Claims

1. A display screen refresh method, characterized in that, The method is applied to an electronic device, the electronic device including a display screen, the display screen being an OLED display screen or an LCD display screen, the method comprising: At the first moment, acquire the image data of the first layer; At a second time point, image data of the second layer is acquired, wherein the second time point is after the first time point; A first region of interest (ROI) in the second layer is determined based on the image data of the first layer and the image data of the second layer, wherein the first ROI includes a first region, which is the region in the second layer where the image data of the second layer has changed relative to the first layer; Send the image data of the first ROI to the display screen; Send the location information of the first ROI to the display screen, wherein the location information of the first ROI indicates the second region; The display screen refreshes the image data of the first ROI in the second area.

2. The method according to claim 1, characterized in that, Before sending the image data of the first ROI to the display screen, the method further includes: Based on the image data of the first ROI in the first layer, the image data of the first ROI is obtained through layer compositing.

3. The method according to claim 1 or 2, characterized in that, When the display screen refreshes the image data of the first ROI in the second area, the method further includes: The display screen does not refresh in areas other than the second area.

4. The method according to any one of claims 1-3, characterized in that, The first ROI is specifically the area obtained by extending the first region horizontally to the column edge in the second layer.

5. The method according to any one of claims 1-4, characterized in that, The time interval between the first moment and the second moment is less than the preset time interval.

6. The method according to any one of claims 1-5, characterized in that, After acquiring the image data of the second layer, the method further includes: The N Regions of Interest (ROIs) corresponding to the N layers are sequentially acquired. Based on the image data of the N ROIs corresponding to the N layers, the areas indicated by the position information of the N ROIs on the display screen are sequentially refreshed.

7. The method according to claim 6, characterized in that, The method specifically includes: The display screen uses a first frequency to refresh the image data of the first ROI in the second area, and in the... The image data of the N ROIs are refreshed sequentially, indicating the regions indicated by the location information of the N ROIs. Wherein, the first frequency is greater than or equal to the second frequency, and less than or equal to the third frequency, the second frequency is the lowest refresh frequency supported by the electronic device, and the third frequency is the highest refresh frequency supported by the electronic device.

8. The method according to claim 6 or 7, characterized in that, After sequentially obtaining the N ROIs corresponding to the N layers, the method further includes: Obtain non-ROIs, where the non-ROIs are continuous regions in the Nth layer that include a third region, and the third region is the region in the Nth layer other than the first ROI and the N ROIs; The display screen refreshes the image data of the non-ROI in a fourth region, wherein the fourth region is the region indicated by the location information of the non-ROI.

9. The method according to any one of claims 6-8, characterized in that, The method specifically includes: The time interval between acquiring the first ROI and acquiring the Nth ROI corresponds to a second frequency, which is the lowest refresh frequency supported by the electronic device.

10. The method according to any one of claims 6-9, characterized in that, Before obtaining the non-ROI, the method further includes: Among the first ROI and the N ROIs obtained, the interval between any two adjacent ROIs is less than the first time duration.

11. The method according to any one of claims 1-4, characterized in that, Before obtaining the first ROI in the second layer, the method further includes: Obtain the second ROI in the first layer; Send the image data of the second ROI to the display screen; Send the location information of the second ROI to the display screen, wherein the location information of the second ROI indicates the fifth region; The display screen refreshes the image data of the second ROI in the fifth region; If, within a first time period after acquiring the second ROI in the first layer, it is determined that no ROI in other layers has been acquired, then the display screen refreshes the image data of the first layer in all areas.

12. The method according to claim 11, characterized in that, The time interval between the first time point and the second time point is greater than or equal to a preset time interval.

13. The method according to claim 11 or 12, characterized in that, The display screen refreshes the image data of the first layer across the entire area, specifically including: The display screen uses a second frequency to refresh the image data of the first layer in all areas. The second frequency is the lowest refresh rate supported by the electronic device.

14. The method according to any one of claims 11-13, characterized in that, After the display screen refreshes the image data of the first layer across the entire area, the method further includes: Based on the acquired first ROI, the display screen is stopped from refreshing the image data of the first layer in all areas.

15. The method according to any one of claims 1-14, characterized in that, The display screen includes a display driver integrated circuit (DDIC), a random access memory (DDIC RAM) of the DDIC, and a display panel. Send the image data of the first ROI to the DDIC RAM of the display screen; Send the location information of the first ROI to the DDIC of the display screen; Image data of the first ROI is obtained from the DDIC RAM based on the location information of the first ROI through the DDIC. The display panel refreshes the image data of the first ROI in the second area.

16. The method according to any one of claims 1-15, characterized in that, The electronic device also includes a hardware synthesizer (HWC). The HWC is used to first obtain the image data of the first layer, and then to obtain the image data of the second layer. The HWC determines the first ROI in the second layer based on the image data of the first layer and the image data of the second layer.

17. The method according to claim 15 or 16, characterized in that, The electronic device also includes a display driver; Sending the image data of the first ROI to the display screen specifically includes: sending the image data of the first ROI to the display driver through the HWC, and sending the image data of the first ROI to the display screen through the display driver; Sending the location information of the first ROI to the display screen specifically includes: sending the location information of the first ROI to the display driver through the HWC, and sending the location information of the first ROI to the display screen through the display driver.

18. The method according to claim 7, characterized in that, The electronic device further includes a display compositing system SF, a first module, HWC, and a display driver. The display screen uses a first frequency to refresh the image data of the first ROI in the second region. Before sequentially refreshing the image data of the N ROIs in the regions indicated by the position information of the N ROIs, the method further includes: The third frequency is sent from the SF to the HWC, the third frequency is sent from the HWC to the display driver, and the third frequency is sent from the display driver to the display screen. The first module sends the second frequency to the display driver, and the display driver sends the second frequency to the display screen.

19. The method according to any one of claims 11-14, characterized in that, The electronic device further includes a display driver. If, within a first time period after acquiring the second ROI in the first layer, it is determined that no ROIs in other layers have been acquired, the display screen refreshes the image data of the first layer across the entire area. Specifically, this includes: Based on the acquired image data of the second ROI, a timer is started using the display driver. The display driver determines whether the timer's timing duration has reached a first duration. If the timer's timing duration has reached the first duration, the display driver controls the display screen to refresh the entire area of ​​the display screen based on the image data of the first layer. The timer is restarted based on the acquired image data of the new ROI, driven by the display driver.

20. The method according to claim 19, characterized in that, The display screen then refreshes the image data of the first layer across the entire area, specifically including: The display screen is controlled by the display driver, and the entire area of ​​the display screen is refreshed based on the image data of the first layer using a second frequency; the second frequency is the lowest frequency supported by the electronic device.

21. An electronic device, characterized in that, The method includes one or more memories, one or more processors, and a computer program stored on the memories, a display screen, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1-20.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the steps of the method according to any one of claims 1-20.

23. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program is executed by a processor, it performs the steps of the method according to any one of claims 1-20.

Images