Interface updating method and electronic equipment

Abstract

The invention provides an interface updating method and electronic equipment. In the method, when the electronic equipment updates a user interface, all dirty regions in the user interface can be merged into K2 regions of interest (ROI), and after updated contents (also called updated contents) in the regions to be updated are obtained, a screen region for displaying the updated contents in the regions to be updated is refreshed in a screen, so that the user interface can be updated, and the user interface can be updated. The screen area for displaying the content in the other area in the screen does not need to be refreshed, and the state after the previous lightening is continuously kept. By implementing the technical scheme provided by the invention, the power consumption during screen refreshing can be reduced under the scene that only the content in the partial region in the user interface needs to be updated.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an interface updating method using an electronic device and the electronic device. Background Art

[0002] The content displayed on the screen of an electronic device can be called a user interface (hereinafter referred to as an interface). The content in the user interface is updated. In some scenarios, usually only the content in some areas of the user interface needs to be updated, but the content in other areas does not need to be updated.

[0003] When only a portion of the content in the user interface needs to be updated, the electronic device still refreshes all the content in the user interface on the screen to display the updated user interface. Refreshing all the content in the user interface on the screen means that the screen area for displaying the updated content needs to be refreshed, and the screen area for displaying the non-updated content also needs to be refreshed.

[0004] Because the content in a part of the area of ​​the user interface needs to be updated, the entire screen must be refreshed to update the user interface, which will cause the electronic device to consume more power when refreshing the screen. Summary of the invention

[0005] The present application provides an interface update method and an electronic device. In a scenario where only the content in some areas of the user interface needs to be updated, the content in the some areas can be merged to obtain K2 areas to be updated, and the screen areas corresponding to the K2 areas to be updated can be refreshed in the screen, thereby reducing power consumption when refreshing the screen.

[0006] In a first aspect, the present application provides an interface update method, the method comprising: an electronic device displays a user interface on a screen; in response to a request to update the user interface, the update refers to an update of a local area in the user interface; the electronic device determines K2 first areas to be updated, the K2 first areas to be updated include dirty areas in the user interface; wherein the dirty area is a rectangular area occupied by controls that are updated in the user interface; the area of ​​the first area to be updated is smaller than the area of ​​the screen area; K2 is an integer greater than or equal to 1; the electronic device refreshes the K2 first areas to be updated in the screen, and the screen area outside the first areas to be updated is not refreshed.

[0007] In the above embodiment, when the electronic device updates the user interface, it can merge all dirty regions (dirtyrect) in the user interface into K2 regions of interest (ROI) to be updated, and after obtaining the updated content (also called updated content) in the region to be updated, refresh the screen area used to display the updated content in the region to be updated, and the screen area displaying the content in other regions of the screen does not need to be refreshed, and continues to maintain the state after the last lighting. By implementing the technical solution provided by the present application, in the scenario where only the content in some regions of the user interface needs to be updated, the power consumption when refreshing the screen can be reduced.

[0008] In combination with the first aspect, in some embodiments, the width of the first area to be updated is the same as the width of the screen.

[0009] In combination with the first aspect, in some embodiments, K2 is equal to 1, and the first area to be updated is a rectangular area with the smallest area including all dirty areas in the user interface.

[0010] In combination with the first aspect, in some embodiments, the K2 is greater than or equal to 2, the K2 first areas to be updated do not overlap, the width of the first area to be updated is the same as the width of the screen, and the K2 first areas to be updated are arranged sequentially along a first direction, and the K2 first areas to be updated jointly cover the dirty areas in the user interface; in the first direction, the upper edge of the topmost first area to be updated is above the upper edge of the topmost dirty area, and the lower edge of the bottommost first area to be updated is below the lower edge of the bottommost dirty area. The first direction is the direction of progressive scanning from top to bottom of the screen display.

[0011] In combination with the first aspect, in some embodiments, the dirty areas in the user interface are arranged sequentially along a first direction, and the middle dirty areas except the topmost dirty area and the bottommost dirty area are covered by the area to be updated closest to the middle dirty area, and the covering means that the lower edge of the nearest area to be updated is below the lower edge of the middle dirty area or overlaps with the lower edge of the middle dirty area, and the upper edge of the nearest area to be updated is above the upper edge of the middle dirty area or overlaps with the upper edge of the middle dirty area.

[0012] In combination with the first aspect, in some embodiments, the region to be updated that is closest to the middle dirty region includes: the region to be updated that is closest to the center point of the middle dirty region.

[0013] In combination with the first aspect, in some embodiments, the region to be updated that is closest to the middle dirty region includes: the region to be updated with the smallest area after covering the middle dirty region.

[0014] In combination with the first aspect, in some embodiments, K2 is greater than or equal to 2, and when K2 first areas to be updated are arranged sequentially along the first direction, the r1th first area to be updated includes Gr1 extended dirty areas, the width of the extended dirty area is the same as the width of the first area to be updated, and the extended dirty area includes one dirty area, and when one dirty area does not overlap with dirty areas other than one dirty area in the user interface, the height of the extended dirty area is the same as the height of a dirty area, or when the extended dirty area includes at least two overlapping dirty areas, the height of the extended dirty area is the same as the height of the two overlapping dirty areas after overlapping; Gr1 is greater than or equal to 1, and when Gr1 is greater than 1, the Gr1 extended dirty areas do not overlap with each other along the first direction, and the first direction is the direction of progressive scanning from top to bottom of the screen display.

[0015] In combination with the first aspect, in some embodiments, the distance between the Gr1 extended dirty area and the r1th reference line among the K2 reference lines is closer to or equal to the distance between the reference lines among the K2 reference lines except the r1th reference line; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; and the r1th reference line is the r1th reference line after the K2 reference lines are arranged sequentially along the first direction.

[0016] In combination with the first aspect, in some embodiments, the Gr1 extended dirty areas include a first extended dirty area, and the distance between the first extended dirty area and the r1th reference line among the K2 reference lines is closer to or equal to the distance between the extended dirty areas other than the first extended dirty area in the user interface and the r1th reference line; when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are merged with the first extended dirty area to obtain the r1th first area to be updated, the area of ​​the K2 first areas to be updated is smaller than the first area, and the first area is the area of ​​the K2 first areas to be updated obtained when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are merged with the second extended dirty area; the distance between the second extended dirty area and the r2th reference line among the K2 reference lines is closer to the r2th reference line between the extended dirty areas other than the second extended dirty area in the user interface; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; and the r1th reference line is the r1th reference line after the K2 reference lines are arranged sequentially along the first direction.

[0017] In combination with the first aspect, in some embodiments, K2 is greater than or equal to 2, K2 first areas to be updated are arranged sequentially along the first direction, among the K2 first areas to be updated, X0 first areas to be updated are areas including surface windows, and the height of one of the X0 first areas to be updated is the same as the height of a surface window included in an area to be updated; wherein X0 is less than K2; the surface window is a drawable control, and the surface window does not include a drawable control; one of the X0 surface windows is extended to an area with the same width as the first area to be updated, and does not include the surface window; The dirty areas other than a surface window are included, and the surface window does not overlap with the dirty areas other than the surface window; the K2 first areas to be updated include K2-X0 first areas to be updated in addition to X0 first areas to be updated; the K0 first areas to be updated divide the user interface into K0+1 areas excluding the K0 first areas to be updated, and when the K0+1 area is greater than or equal to K2-X0, at most K2-X0 areas among the K0+1 areas include dirty areas, and the K0+1 areas other than the K2-X0 areas do not include dirty areas.

[0018] In combination with the first aspect, in some embodiments, the user interface includes a system window requesting an update, and the system window includes at least one drawable control; before the electronic device merges the dirty areas in the user interface into K2 non-overlapping first areas to be updated, the method also includes: the electronic device treats the rectangular area occupied by the control whose content is to be updated in the system window as a dirty area.

[0019] In combination with the first aspect, in some embodiments, the user interface also includes a surface window requesting an update, the surface window is a drawable control, and the surface window does not include a drawable control; before the electronic device merges the dirty areas in the user interface into K2 non-overlapping first areas to be updated, the method also includes: the electronic device treats the surface window as a control whose content is to be updated, and treats the rectangular area occupied by the surface window as a dirty area.

[0020] In combination with the first aspect, in some embodiments, the control type of the surface window is surfaceview or textureview.

[0021] In combination with the first aspect, in some embodiments, before the electronic device refreshes the K2 first areas to be updated in the screen, the method also includes: the electronic device obtains the updated content in the K2 first areas to be updated; the electronic device refreshes the K2 first areas to be updated in the screen, specifically including: the electronic device refreshes the K2 first areas to be updated in the screen based on the updated content, and the updated content is displayed in the refreshed K2 first areas to be updated.

[0022] In combination with the first aspect, in some embodiments, the electronic device obtains updated content from the content in K2 first areas to be updated, specifically including: the electronic device merges the dirty areas in the user interface into K1 second areas to be updated; wherein K1 is an integer greater than or equal to 1, and when K1 is greater than or equal to 2, the K1 second areas to be updated do not overlap; the content in the K1 second areas to be updated includes the content in the K2 areas to be updated; the electronic device updates the content in the K1 second areas to be updated to obtain updated content in the K1 second areas to be updated; the electronic device obtains updated content from the updated content in the K2 first areas to be updated in the K1 second areas to be updated.

[0023] In the above embodiment, the second area to be updated can be regarded as the DPUROI involved in the following embodiment. The DPUROI is the area to be processed by the DPU of the electronic device when performing image processing. The area of ​​the K1 second areas to be updated is still smaller than the area of ​​the screen. Compared with the scenario in which the DPU processes the entire screen area in the full screen refresh solution, the implementation of the above method can reduce the content of DPU processing.

[0024] In combination with the first aspect, in some embodiments, the electronic device determines K2 first areas to be updated, specifically including:

[0025] The electronic device merges the dirty areas in the system window into K dirty areas; K is the maximum value of K1 and K2; K1 is the number of areas to be updated that the display processing unit of the electronic device supports processing, and K2 is the number of areas to be updated that the display driver chip of the electronic device supports processing; the electronic device performs width expansion based on the K dirty areas to obtain K third areas to be updated whose width is the same as the width of the screen; the electronic device obtains a fourth area to be updated whose width is the same as the width of the screen based on the surface window, and the height of the fourth area to be updated is the same as the surface window; the electronic device merges Y×K third areas to be updated and X fourth areas to be updated into K2 first areas to be updated, where Y is the number of system windows and X is the number of surface windows.

[0026] In the above embodiment, the third area to be updated can be understood as the area to be updated with the same width as the screen determined based on the system window in the following embodiment. The fourth area to be updated can be understood as the area to be updated with the same width as the screen determined based on the surface window in the following embodiment. In the case where the contents in Y×K third areas to be updated and X fourth areas to be updated can be displayed on the screen, the Y×K third areas to be updated and X fourth areas to be updated can be understood as K11 ROIs involved in the following embodiment.

[0027] In combination with the first aspect, in some embodiments, the electronic device merges Y×K third areas to be updated and X fourth areas to be updated into K2 first areas to be updated, specifically including: the electronic device determines K2 Y-axis reference lines in the screen; the electronic device determines the K2 areas to be updated that are closest to the K2 Y-axis reference lines among the Y×K third areas to be updated and the X fourth areas to be updated; the electronic device merges the areas to be updated other than the K2 areas to be updated among the Y×K third areas to be updated and the X fourth areas to be updated with the K2 areas to be updated in sequence according to a preset rule to obtain K2 first areas to be updated, so that the rth first area to be updated among the K2 first areas to be updated is a rectangular area with the smallest area including Gr areas to be updated, the distance between Gr and the rth area to be updated among the K2 areas to be updated is closer than the distance between Gr and the area to be updated among the K2 areas to be updated, and the area to be updated among the Gr areas to be updated is the third area to be updated or the fourth area to be updated.

[0028] In combination with the first aspect, in some embodiments, the electronic device merges Y×K third areas to be updated and X fourth areas to be updated into K2 first areas to be updated, specifically including: when it is determined that the X fourth areas to be updated do not overlap with the Y×K third areas to be updated, X is less than K2, and the X fourth areas to be updated divide the user interface into K+1 areas that do not include the X fourth areas to be updated, and the K+1 area is greater than or equal to K2-X, at most K2-X areas in the K+1 areas include the third area to be updated, and areas other than K2-X areas in the K+1 areas do not include dirty areas, the electronic device uses the X fourth areas to be updated as the X first areas to be updated; when the Y×K third areas to be updated are greater than or equal to In the case of K2-X, the electronic device determines K2-X Y-axis reference lines in the screen; the electronic device determines the K2-X areas to be updated that are closest to the K2-X Y-axis reference lines among the Y×K third areas to be updated; the electronic device merges the areas to be updated except the K2-X areas to be updated among the Y×K third areas to be updated with the K2-X areas to be updated in sequence according to preset rules to obtain K2-K first areas to be updated, so that the cth first area to be updated among the K2-X first areas to be updated is the rectangular area with the smallest area including the Gc third areas to be updated, and the distance between the Gc third areas to be updated and the cth area to be updated among the K2-X areas to be updated is closer than the distance between the Gc third areas to be updated and the areas to be updated among the K2-X areas to be updated.

[0029] In the above embodiment, the electronic device can use the fourth area to be updated as the first area to be updated, that is, place the surface window separately in a first area to be updated for processing. Since the surface window is usually used to draw videos and game screens, this can facilitate the electronic device to process the content such as videos and game screens for the first area to be updated including the surface window.

[0030] In combination with the first aspect, in some embodiments, before the electronic device merges Y×K third areas to be updated and X fourth areas to be updated into K2 first areas to be updated, the method further includes: the electronic device determines that the Y×K third areas to be updated and X fourth areas to be updated are displayable in the updated user interface.

[0031] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method implemented in the first aspect.

[0032] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method implemented in the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method implemented in the first aspect.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method implemented in the first aspect.

[0035] It is understandable that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, the chip system provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, other beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 An exemplary scenario is shown in which only content in a part of the area of ​​the user interface needs to be updated;

[0037] Figure 2 A schematic diagram of an electronic device update interface in a solution is shown;

[0038] Figure 3A as well as Figure 3B A schematic diagram of an electronic device update interface in another solution is shown;

[0039] Figure 4A as well as Figure 4B A schematic diagram showing an electronic device determining a dirty area in a user interface;

[0040] Figure 5A-Figure 5C A schematic diagram showing an electronic device determining an area to be updated in a user interface;

[0041] Fig. 6A as well as Figure 6B It shows a schematic diagram of an electronic device updating the content in the area to be updated to obtain an updated user interface;

[0042] Fig. 7A as well as Figure 7B Another schematic diagram showing an electronic device determining an area to be updated in a user interface;

[0043] Figure 7C as well as Fig.7D A schematic diagram showing an electronic device obtaining K2 first areas to be updated based on each dirty area in a user interface;

[0044] Figure 8 A schematic diagram of the software and hardware structure involved in updating the user interface of an electronic device is shown;

[0045] Fig. 9 A diagram showing the interaction of software and hardware modules involved in updating the user interface of an electronic device;

[0046] Fig.10 An example diagram showing a dirty region represented by a dirty region based on the position of a system window is shown;

[0047] Fig.11 An exemplary flow chart for determining K window-relative ROIs in a system window is shown;

[0048] Fig.12 A schematic diagram of determining K ROIs relative to the window in the system window is shown;

[0049] Fig.13 A schematic diagram showing a ROI relative to a window is expanded to obtain an ROI with the same width as the screen;

[0050] Fig.14 An exemplary flow chart for determining K1 DPU ROIs in a user interface is shown;

[0051] Fig.15An exemplary schematic diagram of determining K1 DPU ROIs in a user interface is shown;

[0052] Fig.16 A schematic diagram showing the merging of two regions is shown;

[0053] Fig.17 Another exemplary schematic diagram of determining K1 DPU ROIs in a user interface is shown;

[0054] Fig.18 A schematic diagram showing a user interface update through hardware;

[0055] Fig.19 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0057] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0058] In a scenario where only the content in some areas of the user interface needs to be updated, in order to reduce the power consumption when the screen is refreshed, in one solution, when the electronic device updates the user interface (referred to as the interface), all dirty areas (dirtyrect) in the user interface can be merged into a region of interest (ROI) to be updated. After obtaining the updated content (also referred to as the updated content) in the region to be updated, the screen area used to display the updated content in the region to be updated can be refreshed on the screen. The screen area that displays the content in other areas of the screen does not need to be refreshed, and continues to maintain the state after the last lighting. In this way, the power consumption of screen refresh can be reduced when the user interface is updated. Among them, the dirty area refers to the rectangular area occupied by the control whose content is to be updated on the screen, or the dirty area refers to the rectangular area occupied by the control that is updated on the screen. In some possible cases, the rectangular area occupied by a control is the minimum area that can "surround" the control.

[0059] It should be understood that all contents in the user interface are displayed through controls. Common controls with updateable contents may include window controls (referred to as windows), picture controls, progress bar controls (referred to as progress bars), and the like.

[0060] Figure 1 An exemplary scenario is shown in which only content in a partial area of ​​the user interface needs to be updated.

[0061] like Figure 1 As shown in , electronic device 1 can be a mobile phone, and electronic device 2 can be a tablet computer. All the contents displayed on the screen of electronic device 1 can be regarded as a user interface, and the contents to be updated in the user interface displayed by the electronic device 1 include the contents in area 101, and other contents are not updated. Then in this scheme, when the interface is updated, after electronic device 1 obtains the updated content (content A1) in area 101, it only needs to refresh the screen area displaying content A1 on the screen. All the contents displayed on the screen of electronic device 2 can be regarded as a user interface, and the contents to be updated in the user interface displayed by the electronic device 2 include the contents in area 102, and other contents are not updated. Then in this scheme, when the interface is updated, after electronic device 2 obtains the updated content (content A2) in area 102, it only needs to refresh the screen area displaying content A2 on the screen based on content A2.

[0062] It should be understood here that the electronic device may be other electronic devices besides mobile phones and tablet computers, such as watches, etc.

[0063] Figure 2 A schematic diagram of an electronic device update interface in a solution is shown.

[0064] like Figure 2 As shown in (1), in the user interface 1, the controls whose contents are to be updated include a window 201, a comment control 202, a picture control 203, and a progress bar 204. The window 201 is included in the dirty area 1a, the comment control 202 is included in the dirty area 2a, the picture control 203 is included in the dirty area 3a, and the progress bar 204 is included in the dirty area 4a. Figure 2 As shown in (3), the electronic device can Figure 2 The four dirty regions shown in (2) are merged into a region to be updated (ROI) with the same width as the screen. Figure 2 The region to be updated (ROI) shown in (3) may be a rectangular region with the smallest area including the four dirty regions and the same width as the screen width. The four dirty regions may be referred to as dirty regions to be merged.

[0065] The reason for merging all dirty areas into one area to be updated is that the display driver integrated circuit (DDIC) that controls the screen display refreshes the screen area by area. In this solution, DDIC can only support refreshing one area in the screen. After one screen area is refreshed, other screen areas will not be refreshed. At this time, the area to be updated involved in this solution can also be understood as the DDIC area to be updated (ROI).

[0066] The reason why the width of the DDIC to-be-updated area is the same as the screen width includes: when the display driver chip refreshes the screen, it refreshes the screen line by line, wherein the length of a line is fixedly equal to the width of the screen.

[0067] It should be understood here that the aforementioned fixed length of a row is equal to the width of the screen because of the hardware design. However, in some possible cases, when the display driver chip refreshes the screen, it refreshes it row by row. But the length of a row may not be fixed to be the same as the width of the screen. In this way, the width of the DDIC area to be updated may not be the same as the width of the screen. At this time, the DDIC area to be updated may refer to the rectangular area with the smallest area including the dirty area to be merged. This situation will not be described later, and the following text will take the example of a fixed length of a row being equal to the width of the screen for explanation.

[0068] In a scenario where only the content in some areas of the user interface needs to be updated, in order to further reduce the power consumption when the screen is refreshed, an interface update method is proposed. In this method, the DDIC area to be updated in the user interface can also be expanded from one to multiple (at least two). The electronic device supports calculating K2 DDICROIs in a preset manner based on each dirty area in the user interface. The electronic device updates the content in K2 DDICROIs to obtain the updated content in K2 DDICROIs. Subsequently, the screen area used to display the updated content in K2 DDICROIs is refreshed in the screen. The screen area that displays the content in other areas of the screen does not need to be refreshed, and continues to maintain the state after the last lighting. Among them, the updated content in K2 DDICROIs can be understood as the result after the content in K2 DDICROIs is updated. Among them, K2 is an integer greater than or equal to 1. When K2 is an integer greater than or equal to 2, the reason why the screen area in the screen that supports refreshing the updated content in K2 DDICROIs includes: compared with the above solution, the execution logic of DDIC is changed, and DDIC is expanded from one screen area in the original screen that supports refreshing to K2 screen areas in the screen that supports refreshing. When K2 is equal to 1, the method for updating the interface of the electronic device is the same as the above solution. Figure 2 The methods involved in the above are the same as those in the related contents and will not be repeated here.

[0069] Figure 3A as well as Figure 3B A schematic diagram of an electronic device update interface in another solution is shown.

[0070] Figure 3A The figure shows a schematic diagram of an electronic device updating interface when K2 is equal to 2.

[0071] like Figure 3A Zhong (1) and Figure 3A As shown in (2), the dirty areas to be updated in the user interface 1 may include a dirty area 1a, a dirty area 2a, a dirty area 3a, and a dirty area 4a. Figure 3A Zhong (1) and Figure 3A (2) and the above Figure 2 (1) and Figure 2 The above is the same as (2) and will not be repeated here.

[0072] Electronic devices can Figure 3A The four dirty regions shown in (2) are merged into two regions to be updated (ROIs), which are denoted as DDICROI[a1] and DDICROI[a2]. Figure 3AAs shown in (3), DDICROI[a1] can be a rectangular area with the smallest area including dirty areas 1a and 2a and the same width as the screen width, and DDICROI[a2] can be a rectangular area with the smallest area including dirty areas 3a and 4a and the same width as the screen width.

[0073] After obtaining the updated content (also referred to as updated content) in DDICROI[a1] and DDICROI[a2], the electronic device refreshes the screen area used to display the updated content in DDICROI[a1], and refreshes the screen area used to display the updated content in DDICROI[a2]. The pixels displaying the content in other areas of the screen do not need to be refreshed, and continue to maintain the state after the last lighting. In this way, the electronic device can complete the update of user interface 1 and obtain user interface 2.

[0074] Compared with the method of merging the dirty area 1a, the dirty area 2a, the dirty area 3a and the dirty area 4a into one DDICROI, the method of merging the dirty area 1a, the dirty area 2a, the dirty area 3a and the dirty area 4a into two DDICROIs here can make the pixels used to display the content in the area 301 do not need to be refreshed, and the power consumption during screen refresh can be further reduced on the basis of the aforementioned merging into one DDICROI.

[0075] Figure 3B The figure shows a schematic diagram of an electronic device updating interface when K2 is equal to 3.

[0076] like Figure 3B Zhong (1) and Figure 3B As shown in (2), the dirty areas to be updated in the user interface 1 may include a dirty area 1a, a dirty area 2a, a dirty area 3a, and a dirty area 4a. Figure 3B Zhong (1) and Figure 3B (2) and the above Figure 2 (1) and Figure 2 The above is the same as (2) and will not be repeated here.

[0077] Electronic devices can Figure 3B The four dirty regions shown in (2) are merged into three regions to be updated (ROIs), which are denoted as DDICROI[b1], DDICROI[b2], and DDICROI[b3]. Figure 3B DDICROI[b1] shown in (3) can be a rectangular area including dirty area 1a and dirty area 2a with the smallest area and the same width as the screen width, and DDICROI[b2] can be a rectangular area including dirty area 3a to dirty area 4a with the smallest area and the same width as the screen width.

[0078] After obtaining the updated content (also referred to as updated content) in DDICROI[b1], DDICROI[b2], and DDICROI[b3], the electronic device refreshes the screen area used to display the updated content in DDICROI[b1], DDICROI[b2], and DDICROI[b3], and the pixels displaying the content in other areas of the screen do not need to be refreshed, and continue to maintain the state after the last lighting. In this way, the electronic device can complete the update of user interface 1 and obtain user interface 2.

[0079] Compared with the method of merging the dirty area 1a, the dirty area 2a, the dirty area 3a and the dirty area 4a into one DDICROI, the method of merging the dirty area 1a, the dirty area 2a, the dirty area 3a and the dirty area 4a into three DDICROIs here can make the pixels used to display the contents in area 301 and area 302 do not need to be refreshed, and the power consumption during screen refresh can be further reduced on the basis of the aforementioned merging into one DDICROI.

[0080] In some possible cases, the update of the user interface belongs to the category of image processing. Image processing is a complicated matter. The electronic device can assign different stages of image processing to different hardware or software. That is, different hardware or software in the electronic device divides the work and completes the user interface update together. In some possible cases, the role of DDIC is to refresh the corresponding screen area in the screen based on the updated content in K2 DDIC ROIs to achieve the update of the user interface. The operation of updating the content in K2 DDICROIs can be completed by the display processing unit (DPU) in the electronic device. Here, the area to be updated supported by the DPU (also referred to as the DPU area to be updated or DPUROI) and the area to be updated supported by the DDIC (DDICROI) can be the same or different. Among them, the area to be updated supported by the DPU can include K1 DPU areas to be updated. Among them, K1 is an integer greater than or equal to 1.

[0081] When K1 is equal to K2, the DPUROI supported by the DPU and the DDICROI supported by the DDCI are the same, and the electronic device can directly use the K2 DDICROI as the K1 DPUROI. Subsequently, the electronic device can update the content of the K1 DPUROI through the DPU. The electronic device updating the content of the K1 DPUROI through the DPU can be regarded as an implementation method of the aforementioned electronic device updating the content of the K2 DDICROI.

[0082] When K1 is less than K2, the DPUROI supported by the DPU and the DDICROI supported by the DDCI are different, and the content in the K1 DPUROI includes the content in the K2 DDICROI supported by the DDCI. In the process of updating the content in the K1 DPUROI through the DPU, the electronic device also updates the content in the K2 DDICROI. Then, the electronic device can obtain the updated content in the K2 DDICROI from the updated content in the K1 DPUROI.

[0083] It should be understood here that, in order to avoid problems such as repeated updates, there is no overlap between any two DPUROIs in the aforementioned K1 DPUROIs, and there is no overlap between any two DDICROIs in the K2 DDICROIs.

[0084] In the aforementioned interface updating method, the process of the electronic device determining the dirty area in the user interface can refer to the following Figure 4A as well as Figure 4B Description.

[0085] The electronic device may determine a dirty region included in at least one type of window in the user interface as a dirty region included in the user interface in different ways.

[0086] In some possible cases, the window type may include a system window or a surface window. Among them, the system window refers to a window drawn based on the system's image rendering engine, and the electronic device records the drawing control tree of the system window. The information of each control in the system window (including whether the control needs to be updated, the position of the control in the system window, etc.) can be determined based on the drawing control tree. The surface window refers to a window that is not drawn based on the system's image rendering engine. Generally speaking, in addition to the system's image rendering engine that can draw windows, modules such as game engines and image codecs can also draw surface windows. Common surface windows may include surfaceview or textureview, etc. The surface window can usually be used to display images or videos.

[0087] In some possible cases, the surface window itself is a drawable control, and the surface window does not include drawable controls. Surface windows are generally used to draw image content such as videos and game screens. Drawable controls are controls whose content can be updated. However, the system window can include at least one drawable control inside.

[0088] like Figure 4AAs shown, the user interface 1 may include system windows (system window 1 and system window 2) and surface window 1. Here, the example that system window 1 and system window 2 have dirty areas, and system window 2 does not have a dirty area is used for description.

[0089] After receiving the request to update the system window 1 and the request to update the surface window 1 , the electronic device may determine the dirty area in the system window 1 and the dirty area in the surface window 1 as the dirty area in the user interface 1 .

[0090] In some possible implementations, the electronic device determines the control to be updated in the system window 1 based on the drawn control tree, and uses the dirty area where the control to be updated is located as the dirty area in the system window 1. Figure 4B As shown, the dirty area in the system window 1 includes the dirty area 11, the dirty area 12 and the dirty area 13. The electronic device can use the entire area (dirty area 21) occupied by the surface window 1 as the dirty area of ​​the surface window 1.

[0091] In the above-mentioned interface updating method, the electronic device updates the content in K2 DDICROIs, and the relevant content of obtaining the updated content in K2 DDCIROIs can refer to the following Figure 5A-Figure 5C as well as Fig. 6A as well as Figure 6B Description.

[0092] Before the electronic device updates the content in the K2 DDICROIs, it may first determine the K1 DPUROIs in the user interface and the K2 DDICROIs in the user interface. The content in the K1 DPUROIs in the user interface includes the content in the K2 DDICROIs.

[0093] The process of calculating the K1 DPUROIs in the user interface by the electronic device can refer to the following Figure 5A as well as Figure 5B Description.

[0094] Here, the aforementioned user interface 1, system window 1 and surface window 1, K1=1 and K2=2 are taken as examples for explanation.

[0095] The electronic device may first merge the dirty areas in the system window 1 to obtain K areas to be updated relative to the window in the system window 1. Among them, K is equal to the maximum value among K1 and K2. Then the electronic device merges the K areas to be updated relative to the window in the system window 1 and the area to be updated (ROI) in the surface window 1 to obtain K1 DPUROIs in the user interface 1. Among them, the detailed process of the electronic device merging the dirty areas in the system window 1 to obtain K areas to be updated relative to the window can refer to the following description of step S102a, which will not be repeated here.

[0096] like Figure 5A As shown, the electronic device can merge the dirty areas 11-13 in the system window 1 to obtain two regions to be updated (ROIs) relative to the window in the system window 1, including ROI11a and ROI12a. In addition, the electronic device uses the dirty area (dirty area 21) corresponding to the surface window 1 as a region to be updated (ROI) relative to the window of the surface window 1, including ROI21a. The region to be updated relative to the window in the surface window 1 is still included in the surface window. Based on this, the electronic device can determine K+1 regions to be updated relative to the window in the user interface 1, including ROI11a, ROI11b and ROI21a.

[0097] Subsequent, such as Figure 5B As shown in (1), the electronic device expands K+1 regions to be updated relative to the window in the user interface 1 into regions to be updated relative to the screen (ROI), thereby obtaining K+1 regions to be updated relative to the screen. Here, expanding a region to be updated relative to the window into a region to be updated relative to the screen means: determining a region (region to be updated relative to the screen) with the smallest area and the same width as the screen that includes the region to be updated relative to the window. Here, K is equal to 2, K1 is equal to 1, as Figure 5B As shown in (1), the electronic device can determine that the user interface 1 includes three regions to be updated relative to the screen (ROI21b, ROI11b, and ROI12b). Then, the electronic device merges the K+1 regions to be updated relative to the screen into K1 DPU regions to be updated (ROI). Here, K is equal to 2, K1 is equal to 1, as shown in FIG. Figure 5B As shown in (2), the electronic device may determine that the user interface includes one DPU ROI (DPU ROI11a).

[0098] In some possible cases, expansion may also be understood as width expansion, that is, the width of the region is expanded, but the height of the region does not change.

[0099] In some possible cases, the expanded area (eg, the area to be updated relative to the screen) may be referred to as an expanded dirty area.

[0100] The detailed process of the electronic device merging K+1 regions to be updated relative to the screen into K1 DPU regions to be updated (ROI) can be referred to the following description of step S105a and related contents, which will not be described in detail here.

[0101] The process of calculating the K2 DDICROIs in the user interface of the electronic device can refer to the following Figure 5A as well as Figure 5C Description.

[0102] Here, the aforementioned user interface 1, system window 1 and surface window 1, K1=1 and K2=2 are taken as examples for explanation.

[0103] The electronic device first obtains K areas to be updated relative to the window in the user interface 1 based on the dirty areas in the system window 1 and the dirty areas in the surface window 1. This process can refer to the above Figure 5A The description of the related contents will not be repeated here.

[0104] Subsequent, such as Figure 5C As shown in (1), the electronic device can determine that the user interface 1 includes 3 (K+1) regions to be updated relative to the screen (ROI21b, ROI11b, and ROI12b). Then, the electronic device merges the 3 regions to be updated relative to the screen into 1 (K1) DPU region to be updated (ROI). Figure 5C As shown in (2), the electronic device can determine that the user interface includes a DPU ROI (DPU ROI11a). Figure 5C For the relevant content of (1), please refer to the above Figure 5C The description of (1) will not be repeated here.

[0105] The detailed process of the electronic device merging K+1 regions to be updated relative to the screen into K2 DDIC regions to be updated (ROIs) can be referred to the following description of step S105b and related contents, which will not be described in detail here.

[0106] After determining K1 DPUROIs and K2 DDICROIs in the user interface, the electronic device may update the content in the K1 DPUROI (and also update the content in the K2 DDICROIs), and further obtain updated content in the K2 DDCIROIs.

[0107] For details about this process, please refer to the following Fig. 6A as well as Figure 6B Description.

[0108] First, if Fig. 6A As shown in (1), the electronic device updates the content in the dirty area of ​​the system window 1, and sets the area covered by the surface window 1 in the system window 1 of the user interface 1 to be transparent, thereby obtaining the updated system window 1. Fig. 6A As shown in (2), the electronic device can also update the content in the surface window 1 to obtain an updated surface window 1.

[0109] Then, the electronic device may blend the updated system window and the updated surface window to obtain a blended window. In the blended window, the updated surface window 1 is displayed in the transparent area of ​​the updated system window 1. Then, the electronic device may obtain the content in the area corresponding to the K1 DPUROIs from the blended window as the updated DPU image. Fig. 6A As shown in (3), the electronic device obtains K1 (1) exemplary DPU images based on the updated surface window 1 and the updated system window 1. When K1 is equal to 1, the area of ​​the DPU image is equal to the sum of the areas of the DPUROIs, and the area occupied by the DPU image on the screen is the same as the area occupied by the DPUROI on the screen.

[0110] The electronic device then obtains a DDIC image included in the DPU image based on the positions of the K2 DDICROIs on the screen. The DDIC image may include the updated content in the K2 DDICROIs. The area of ​​the DDIC image is equal to the sum of the areas of the K2 DDICROIs, and the area occupied by the DDIC image on the screen is the same as the area occupied by the K2 DDICROIs on the screen. For example, Figure 6B As shown in (1), when K2 is equal to 2, the DDIC image obtained by the electronic device includes the updated content in the two DDICROIs. Then the electronic device can use the DDIC image to update the content in the area occupied by T2 DDIC ROIs in the user interface 1 to obtain the user interface 2. That is, the electronic device refreshes the screen area used to display the updated content in the K2 DDICROIs on the screen, and keeps the state of the pixels displaying the content in other areas on the screen after the last lighting, so that the electronic device can display the user interface 2 on the screen.

[0111] It should be understood that the number of to-be-updated areas supported for processing in the DPU, K1, is equal to 1, and the number of to-be-updated areas supported for processing in the DDIC, K2, is equal to 2, for example. In actual situations, other values ​​may be set according to the processing conditions of the hardware, for example, K1 is equal to 2, K2 is equal to 3, etc. This is not limited in the embodiments of the present application.

[0112] Fig. 7A as well as Figure 7B Another schematic diagram showing an electronic device determining an area to be updated in a user interface.

[0113] Here, the aforementioned user interface 1, system window 1 and surface window 1, K1=2 and K2=3 are used as examples for description. Another example of the electronic device determining the DPUROI and DDICROI in the user interface 1 is described.

[0114] like Fig. 7A As shown, the electronic device can first merge the dirty regions in the system window 1 to obtain K (here K is equal to 3) ROIs relative to the window in the system window 1, including ROI11c, ROI12c and ROI13c. And the electronic device uses the dirty region (dirty region 21) corresponding to the surface window 1 as an ROI relative to the window of the surface window 1, including ROI21b. For a detailed description of the process, please refer to the aforementioned Figure 5A The description is not repeated here.

[0115] It should be understood that ROI21b here is the same as ROI21a mentioned above. Since the number of dirty regions included in system window 1 is equal to the maximum value of K2 and K1, the three dirty regions in system window 1 can be directly used as the three ROIs in system window 1 relative to the window.

[0116] Subsequent, such as Figure 7B As shown in (1), the electronic device expands K+1 regions to be updated relative to the window in the user interface 1 (including ROI21c, ROI11c, ROI12c, and ROI13c) into regions to be updated relative to the screen (ROI), thereby obtaining K+1 regions to be updated relative to the screen (including ROI21d, ROI11d, ROI12d, and ROI13d). Figure 7B As shown in (2), the electronic device merges the four regions to be updated relative to the screen into two DPU regions to be updated (ROI). For a detailed description of this process, please refer to the aforementioned Figure 7B The description is not repeated here.

[0117] And, if Figure 7B As shown in (3), the electronic device can also merge the four to-be-updated regions relative to the screen into three DDIC to-be-updated regions (ROIs). For a detailed description of this process, please refer to the aforementioned Figure 5C The description is not repeated here.

[0118] The DDICROI involved in the foregoing content may also be referred to as the first area to be updated.

[0119] Figure 7C as well as Fig.7D A schematic diagram is shown in which the electronic device obtains K2 first areas to be updated based on the dirty areas in the user interface.

[0120] Combine the following Figure 7C as well as Fig.7D Detailed Description The electronic device obtains relevant contents of K2 first areas to be updated based on the dirty areas in the user interface.

[0121] The electronic device displays a user interface on the screen. In response to a request to update the user interface, the electronic device determines K2 first areas to be updated, and the K2 first areas to be updated include dirty areas in the user interface. Based on the aforementioned related content, it can be seen that the area of ​​the first area to be updated is smaller than the area of ​​the screen area; K2 is an integer greater than or equal to 1; when K2 is greater than or equal to 2, the K2 first areas to be updated do not overlap. Then the electronic device refreshes the K2 first areas to be updated in the screen, and the screen area outside the first areas to be updated is not refreshed.

[0122] It should be understood here that the update involved in the request to update the user interface refers to the update of a local area in the user interface. For example, the request to update the user interface may include a request to update a system window or a request to update a surface window.

[0123] In some possible cases, generally speaking, the width of the first area to be updated is the same as the width of the screen.

[0124] The performance of the K2 first to-be-updated regions in different situations is described in detail below.

[0125] When K2 is equal to 1, the electronic device may determine that the first area to be updated is a rectangular area with the smallest area including all dirty areas in the user interface. Figure 7C As shown in (1), the user interface may include dirty area 1, dirty area 2, dirty area 3, dirty area 4, and dirty area 5. The electronic device may merge dirty area 1-dirty area 5 into a first area to be updated (e.g., area to be updated 11), and the area to be updated 11 is a rectangular area with the smallest area including dirty area 1-dirty area 5.

[0126] When K2 is greater than or equal to 2, K2 first areas to be updated are arranged in sequence along the first direction; the upper edge of the top first area to be updated is above the upper edge of the top dirty area (above includes overlapping with the upper edge of the top dirty area), and the lower edge of the bottom first area to be updated is below the lower edge of the bottom dirty area (below includes overlapping with the lower edge of the bottom dirty area); wherein, the top dirty area is the top dirty area of ​​all dirty areas in the user interface along the first direction, and the bottom dirty area is the bottom dirty area after all dirty areas in the user interface are arranged in sequence along the first direction. The first direction is the direction of scanning from top to bottom line by line in the screen display, not the direction of scanning a single line from left to right. For example, refer to Figure 7C As shown in (3), the first dirty area is dirty area 1, the second dirty area is dirty area 5, and after K2 first areas to be updated (including areas to be updated 21 and areas to be updated 22) are arranged along the first direction, the top first area to be updated is area to be updated 21, and the bottom first area to be updated is area to be updated 22. The upper edge of area to be updated 21 overlaps with the upper edge of dirty area 1, and the lower edge of area to be updated 21 overlaps with the lower edge of dirty area 5. For the case of K=3, please refer to Figure 7C As shown in (4), the above-mentioned contents are also satisfied and will not be described again here.

[0127] When the dirty areas in the user interface do not overlap, the dirty areas in the user interface are arranged sequentially along the first direction, and the middle dirty areas except the top dirty area and the bottom dirty area are covered by the area to be updated closest to the middle dirty area, and the covering means that the lower edge of the nearest area to be updated is below the lower edge of the middle dirty area or coincides with the lower edge of the middle dirty area, and the upper edge of the nearest area to be updated is above the upper edge of the middle dirty area or coincides with the upper edge of the middle dirty area.

[0128] In some possible cases, the area to be updated that is closest to the middle dirty area includes: the area to be updated that is closest to the center point of the middle dirty area.

[0129] In some other possible cases, the region to be updated that is closest to the middle dirty region includes: the region to be updated with the smallest area after covering the middle dirty region.

[0130] In the case where K2 is greater than or equal to 2, when K2 first areas to be updated are arranged in sequence along the first direction, the r1th first area to be updated includes Gr1 extended dirty areas, the width of the extended dirty areas is the same as the width of the first area to be updated, and the extended dirty areas include one dirty area, and when the one dirty area does not overlap with dirty areas other than the one dirty area in the user interface, the height of the extended dirty area is the same as the height of the one dirty area, or when the extended dirty area includes at least two overlapping dirty areas, the height of the extended dirty area is the same as the height of the two overlapping dirty areas after overlapping; Gr1 is greater than or equal to 1, and when Gr1 is greater than 1, the Gr1 extended dirty areas do not overlap with each other along the first direction. For example, referring to Figure 7C The area to be updated 22 shown in (3) includes an extended dirty area 1 and an extended dirty area 2. The extended dirty area 1 includes a dirty area 5, which does not overlap with other dirty areas, and the height of the extended dirty area 1 is the same as that of the dirty area 5. The dirty area 3 and the dirty area 4 overlap and are merged into the extended dirty area 2, and the height of the extended dirty area is the same as the height after the dirty area 3 and the dirty area 4 overlap.

[0131] In some possible implementations, when K2 is greater than or equal to 2, the process of the electronic device determining K2 first areas to be updated based on each dirty area in the user interface may include: the electronic device may determine K2 reference lines (perpendicular to the first direction) in the screen. K2 extended dirty areas that are closest to the K2 reference lines in the extended dirty areas determined by each dirty area. Then, the extended dirty areas other than the K2 extended dirty areas in the extended dirty areas determined by each dirty area are sequentially added to the K2 extended dirty areas and merged with the K2 extended dirty areas to obtain K2 first areas to be updated. After the merger is completed, each of the K2 first areas to be updated may include at least one extended dirty area.

[0132] In some possible cases, when K2 is greater than or equal to 2, when the K2 first areas to be updated are arranged in sequence along the first direction, the r1th first area to be updated includes the Gr1th extended dirty area, and the distance between the Gr1th extended dirty area and the r1th reference line among the K2 reference lines is closer to or equal to the distance between the reference lines among the K2 reference lines except the r1th reference line; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; and the r1th reference line is the r1th reference line after the K2 reference lines are arranged in sequence along the first direction. Figure 7C As shown in (2), when K2=2, the K2 first areas to be updated include (area to be updated 21 and area to be updated 22). For the description of the above related contents, please refer to Figure 7C (3) will not be repeated here.

[0133] It should be understood here that a region (such as an extended dirty region) being closest to a reference line compared to other reference lines may include that the center of the region is closest to the reference line compared to other reference lines. When the center of the region is equally close to two reference lines, it can be further determined by the upper edge of the region which reference line is closer to the region.

[0134] In some other possible cases, the Gr1 extended dirty areas include the first extended dirty area, and the distance between the first extended dirty area and the r1th reference line among the K2 reference lines is closer or equal to the distance between the extended dirty areas other than the first extended dirty area in the user interface and the r1th reference line; when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are combined with the first extended dirty area to obtain the r1th first area to be updated, the area of ​​the K2 first areas to be updated is smaller than the first area, and the first area is the area of ​​the K2 first areas to be updated obtained when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are combined with the second extended dirty area; the distance between the second extended dirty area and the r2th reference line among the K2 reference lines is closer than the distance between the extended dirty areas other than the second extended dirty area in the user interface and the r2th reference line; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; and the r1th reference line is the r1th reference line after the K2 reference lines are arranged in sequence along the first direction. The reference lines include reference line 1 and reference line 2. like Figure 7C As shown in (4), when K2=3, the K2 first areas to be updated include (area to be updated 31, area to be updated 32 and area to be updated 33). The reference lines include reference line 1, reference line 2 and reference line 3. The description of the above related contents can be Figure 7C (4) will not be repeated here.

[0135] In some other possible implementations, when K2 is greater than or equal to 2, the process of the electronic device determining K2 first areas to be updated based on the dirty areas in the user interface may include: the electronic device may determine that X0 surface windows that meet the conditions are individually included in a first area to be updated, and obtain X0 first areas to be updated, and the height of one of the X0 first areas to be updated is the same as the height of the surface window included in the area to be updated. Among them, X0 is less than K2; the K2 first areas to be updated include K2-X0 first areas to be updated in addition to the X0 first areas to be updated; the K0 first areas to be updated divide the user interface into K0+1 areas that do not include the K0 first areas to be updated, and when the K0+1 area is greater than or equal to K2-X0, at most K2-X0 areas in the K0+1 area include dirty areas, and areas other than K2-X0 areas in the K0+1 area do not include dirty areas. The process of an electronic device obtaining K2-X0 first areas to be updated based on the dirty areas included in the K2-X0 areas includes: the electronic device can determine K2-X0 reference lines (perpendicular to the first direction) in the screen. Determine the K2-X0 extended dirty areas that are closest to the K2-X0 reference lines in the extended dirty areas determined by the dirty areas included in the K2-X0 areas. Then, the extended dirty areas other than the K2-X0 extended dirty areas in the extended dirty areas determined by the dirty areas included in the K2-X0 areas are merged with the K2-X0 extended dirty areas in sequence to obtain K2-X0 first areas to be updated. After the merger is completed, each of the K2 first areas to be updated can include at least one extended dirty area. The performance of the K2-X0 first areas to be updated is the same as that of the aforementioned K2 first areas to be updated, and the aforementioned related content can be referred to and will not be repeated here.

[0136] Among them, the conditions that the surface windows in the X0 first areas to be updated must meet include: an area obtained by extending one of the X0 surface windows to the same width as the first area to be updated does not include a dirty area other than the one surface window, and one of the surface windows does not overlap with a dirty area other than the one surface window.

[0137] When K2 is greater than or equal to 2, K2 first areas to be updated are arranged in sequence along the first direction, among the K2 first areas to be updated, there are X0 first areas to be updated that include surface windows, and the height of one of the X0 first areas to be updated is the same as the height of the surface window included in one area to be updated; wherein X0 is less than K2; the surface window is a drawable control, and the surface window does not include drawable controls; an area obtained by extending a surface window in the X0 surface windows to the same width as the first areas to be updated does not include dirty areas other than one surface window, and one surface window does not overlap with dirty areas other than one surface window; the K2 first areas to be updated include K2-X0 first areas to be updated in addition to the X0 first areas to be updated; the K0 first areas to be updated divide the user interface into K0+1 areas that do not include the K0 first areas to be updated, and when the K0+1 area is greater than or equal to K2-X0, at most K2-X0 areas in the K0+1 area include dirty areas, and areas other than the K2-X0 areas in the K0+1 area do not include dirty areas. like Fig.7D As shown in (1), dirty area 1 is a surface window, and the length and width of the surface meet the condition that it belongs to a first area to be updated alone. Then, the electronic device can determine a first area to be updated based on the surface window, and obtain another first area to be updated based on dirty area 2, dirty area 3, dirty area 4 and dirty area 5. Fig.7D As shown in (2), when K2 is equal to 2, the electronic device can also merge the dirty areas (dirty area 2-dirty area 5) located below the surface window to obtain 1 (K2-1) first areas to be updated (for example, area to be updated 24). At this time, the first area to be updated including the surface window is area to be updated 23. Fig.7D As shown in (3), when K2 is equal to 3, the electronic device can also merge the dirty areas (dirty area 2-dirty area 5) located below the surface window to obtain 2 (K2-1) first areas to be updated (for example, area 35 to be updated and area 36 to be updated). At this time, the first area to be updated including the surface window is area 34 to be updated. Fig.7D If there is a dirty area above the upper edge of the surface window shown in (3), the electronic device can merge the dirty areas above the upper edge into a first area to be updated, and merge the dirty areas below the lower edge of the surface window into another first area to be updated.

[0138] Figure 8 A schematic diagram of the software and hardware structure involved in updating the user interface of an electronic device is shown.

[0139] like Figure 8As shown, the layered architecture divides the hardware and software structure into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the hardware and software structure is divided into four layers, from top to bottom, namely, the application layer, the system library, the kernel layer, and the hardware layer.

[0140] The application layer can include a series of applications, referred to as applications.

[0141] An application may provide a window to display content on a screen, and all the content displayed on the screen may be referred to as a user interface. The windows included in the user interface may come from different applications. A window in a user interface may include a system window, a surface window, etc. For the relevant content of the system window and the surface window, please refer to the relevant content, which will not be repeated here.

[0142] The system library may include an image rendering engine and an image mixing renderer. Common image rendering engines may include a hardware user interface (HWUI), and common image mixing renderers may include HWC (height-width-channel).

[0143] The image rendering engine includes a single-window ROI calculation module, which can be used to determine a dirty area in a system window whose content is to be updated in a user interface, and determine K areas to be updated in the system window relative to the window based on the dirty area in the system window. In addition, the K areas to be updated in the system window relative to the window are sent to the image mixing renderer. The system window whose content is to be updated in the user interface can include one or more.

[0144] The image hybrid renderer includes a multi-window DPUROI calculation module and a multi-window DDICROI calculation module.

[0145] Among them, the multi-window DPUROI calculation module can be used to obtain the area to be updated relative to the window of the surface window whose content is to be updated. It can also be used to obtain Y×K+X areas to be updated relative to the screen obtained by expanding the K areas to be updated relative to the window in the system window and the areas to be updated relative to the window in the surface window. Then, based on the Y×K+X areas to be updated relative to the screen in the user interface, they are merged to obtain K1 DPUROIs. Among them, Y is the number of surface windows and X is the number of system windows.

[0146] The multi-window DDICROI calculation module can be used to obtain the area to be updated relative to the window of the surface window whose content is to be updated. It can also be used to obtain Y×K+X areas to be updated relative to the screen obtained by expanding the K areas to be updated relative to the window in the system window and the areas to be updated relative to the window in the surface window. Then, based on the Y×K+X areas to be updated relative to the screen in the user interface, they are merged to obtain K2 DDICROIs. Among them, Y is the number of surface windows and X is the number of system windows.

[0147] The kernel layer may include a display driver, which may generally be a software display engine (SDE) driver.

[0148] The display driver can be used to respond to a request to update the user interface and drive the hardware to update and display the user interface.

[0149] The hardware layer may include a graphics processing unit (GPU), a display processing unit (DPU), and a display driver chip (DDIC).

[0150] The GPU may be used to update the system window using the dirty area included in the system window to obtain an updated system window, and to update the surface window to obtain an updated surface window.

[0151] The DPU may be used to mix the updated system window and the updated surface window, determine the updated content in K1 DPU to-be-updated areas in the mixed window, and obtain a DPU image. When K1 is equal to 1, the DPU may determine the updated content in 1 DPU to-be-updated area in the mixed window based on the DPUROI update module 1, and obtain a DPU image.

[0152] DDIC can be used to determine the updated content in each of the K2 DDIC to-be-updated areas in the DPU image based on T2 DDIC ROIs to obtain a DDIC image. And the content in the area occupied by the K2 DDIC ROIs in the user interface is updated using the DDIC image to obtain an updated user interface. That is, DDIC is also used to refresh the screen area used to display the updated content in the K2 DDICROIs on the screen, and keep the pixels displaying the content in other areas of the screen in the state after the last lighting, so as to display the user interface 2 on the screen. When K2 is equal to 2, DDIC can update the content in the area occupied by the two DDIC ROIs in the user interface using the DDIC image based on the DDIC ROI update module 1 and the DDIC ROI update module 2 to obtain an updated user interface.

[0153] It should be understood here that the aforementioned content is described in the case where the number of dirty areas in the system window is greater than or equal to the maximum value of K1 and K2, and the contents in the Y×K+X areas to be updated relative to the screen can all be displayed in the updated user interface. In actual situations, the number of dirty areas in the system window (recorded as J) can be less than the maximum value of K1 and K2. In this case, the number of ROIs relative to the window and relative to the screen in the system window obtained by the electronic device is no longer K, but J. Furthermore, the number of ROIs relative to the window and relative to the screen in the system window and the surface window obtained by the electronic device is not Y×K+X in total, but Y×J+X. In the case where the contents in the Y×J+X areas to be updated relative to the screen or the Y×K+X areas to be updated relative to the screen with K11 areas to be updated relative to the screen can all be displayed in the updated user interface, if K11 is less than or equal to K1, the number of DPURIOs determined by the electronic device is not K1 but K11. If K11 is less than or equal to K2, the number of DDICRIOs determined by the electronic device is not K2 but K11.

[0154] In some possible cases, there are many dirty areas in the system window. The electronic device can first merge the dirty areas of the system window and then merge them with the surface window to obtain K1 DPUROIs or R2 DDICROIs. For the relevant content involved in this process, please refer to the following Fig. 9 Description.

[0155] Fig. 9 The diagram shows the interaction between software and hardware modules involved when an electronic device updates a user interface.

[0156] For an exemplary process of updating a user interface of an electronic device, reference may be made to the description of the following steps.

[0157] S101a. The single-window ROI calculation module in the image rendering engine determines at least one dirty region included in the system window.

[0158] An exemplary description of step S101a can be referred to the above Figure 4A as well as Figure 4B The description of getting the dirty area of ​​system window 1 is shown in .

[0159] When a system window or a surface window of the user interface is to be updated, in response to the system window in the user interface requesting an update, the single window ROI calculation module in the image rendering engine executes step S101a to determine at least one dirty region included in the system window.

[0160] The system window refers to the window that is drawn based on the system's image rendering engine. The single-window ROI calculation module can obtain the drawing control tree of the system window. The drawing control tree records the information of each control in the system window (including whether the control needs to be updated, the position of the control in the system window, etc.).

[0161] The single window ROI calculation module can determine the rectangular area occupied by the control whose content is to be updated in the system window as the dirty area based on the drawing control tree, wherein the rectangular area occupied by the control is determined based on the position of the control in the system window.

[0162] In some possible cases, the rectangular areas occupied by the controls to be updated do not intersect with each other, and the number of dirty areas included in the system window is equal to the number of controls to be updated.

[0163] In other possible cases, if the rectangular areas occupied by the controls to be updated intersect, the single window ROI calculation module can merge the rectangular areas occupied by the intersecting controls to be updated into a dirty area, and the merged dirty area is the minimum rectangular area including the intersecting controls to be updated. Non-intersecting rectangular areas are each regarded as dirty areas, and the number of dirty areas included in the system window is less than the number of controls to be updated.

[0164] It can be seen that the dirty areas included in the system window determined by the single-window ROI calculation module do not intersect with each other.

[0165] The single window ROI calculation module may use a dirty region to represent the dirty region based on the position of the system window. Alternatively, the single window ROI calculation module may use a dirty region to represent the dirty region based on the position of the screen.

[0166] The following Fig.10An example of using a dirty region to represent the dirty region based on the position of the system window is described in and related content. The description of using a dirty region to represent the dirty region based on the position of the screen is similar, except that the system window is replaced by the screen, which will not be repeated here.

[0167] Fig.10 An example diagram showing a dirty region represented by a dirty region based on a position of a system window is shown.

[0168] The dirty area is usually expressed using a coordinate system based on the position of the system window, the most common of which is the pixel coordinate system of the system window or window.

[0169] like Fig.10 As shown, in the pixel coordinate system on the system window, the origin (0, 0) is the upper left corner of the system window, the horizontal direction is the X axis, and the vertical direction is the Y axis. The horizontal direction increases to the right, and the vertical direction increases downward. The width and height of the system window are measured in pixels.

[0170] In the pixel coordinate system on the system window, a dirty area can be represented by the coordinates of the two diagonal corners of the dirty area (including the X-axis coordinate and the Y-axis coordinate) based on the position of the system window, usually the coordinates of the upper left corner and the lower right corner of the dirty area. For example, a rectangular area can be represented as (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left corner and (x2, y2) is the coordinate of the lower right corner.

[0171] It should be understood that the left and right involved in the aforementioned coordinate system are two opposite directions, the up and down are also two opposite directions, and the horizontal direction is perpendicular to the vertical direction. This is not a specific reference and should not constitute a limitation on the embodiments of the present application.

[0172] It should also be understood that using the coordinates of the two diagonal corners of a region (e.g., a dirty region) to represent the position of a region based on the system window is one implementation method. In actual situations, other implementation methods may also be used, and the embodiments of the present application do not limit this. For example, the position of a region may also be represented by the coordinates of the center of the region, the width of the region, and the height of the region. The width of the region and the height of the region are represented in pixels.

[0173] S101b. The image mixing renderer determines that the area occupied by the surface window is a dirty area corresponding to the surface window.

[0174] An exemplary description of step S101b can be referred to the aforementioned Figure 4A as well as Figure 4B The description of getting the dirty area of ​​surface window 1 is shown in FIG.

[0175] When the system window or surface window of the user interface is to be updated, in response to the surface window in the user interface requesting an update, the multi-window DPUROI calculation module in the image mixing renderer executes step S101b to determine that the area occupied by the surface window is the dirty area corresponding to the surface window.

[0176] A surface window is a window that is not drawn based on the system's image rendering engine. Generally speaking, in addition to the system's image rendering engine, game engines, image codecs and other modules can also draw surface windows. Common surface windows include surfaceview or textureview. The surface window can usually be used to display images or videos with variable content.

[0177] The multi-window DPUROI calculation module can use the dirty area corresponding to the surface window to represent the surface window based on the screen or based on the position of the surface window. Among them, the method of determining the position of the dirty area corresponding to the surface window based on the screen is similar to the method of determining the position of the dirty area in the system window based on the system window mentioned above, and the description of the system window can be replaced with the screen. The position of the dirty area corresponding to the surface window based on the screen is the same as the position of the surface window based on the screen. The method of determining the dirty area corresponding to the surface window based on the position of the surface window is similar to the method of determining the dirty area in the system window based on the system window mentioned above, and the description of the system window can be replaced with the surface window.

[0178] S102a. The single-window ROI calculation module determines F ROIs relative to the window in the system window based on at least one dirty region in the system window.

[0179] When the number of dirty areas in the system window (denoted as J) is less than or equal to the maximum value of K1 and K2 (denoted as K), the single window ROI calculation module can use the dirty areas in the system window as the ROI relative to the window in the system window. At this time, F is equal to J. When J is greater than the maximum value of K1 and K2 (denoted as K), the single window ROI calculation module can merge the J dirty areas in the system window to obtain K ROIs relative to the window in the system window, and at this time, F is equal to K. K1 is the number of DPUROIs that the DPU supports processing. K2 is the DDICROI that the DDIC supports processing. K is an integer greater than or equal to 2.

[0180] It should be understood that the ROI relative to the window refers to the ROI still included in the system window, and the width of the ROI relative to the window is less than or equal to the width of the system window.

[0181] It should also be understood that the number of dirty regions (denoted as J) in different system windows is different.

[0182] The manner in which the single-window ROI calculation module merges the J dirty regions in the system window into K ROIs relative to the window in the system window includes but is not limited to the following preset manner 11.

[0183] Fig.11 An exemplary flow chart for determining K window-relative ROIs in a system window is shown.

[0184] For relevant contents related to preset method 11, please refer to the following Fig.11 Description of steps S201-S203 in .

[0185] S201. Determine K Y-axis reference lines in the system window.

[0186] In some possible cases, the Y-axis reference line is a line perpendicular to the Y-axis, which may be simply referred to as a reference line.

[0187] The K Y-axis reference lines in the system window can divide the system window into K-1 areas. The value of the i-th Y-axis reference line among the K Y-axis reference lines is yi = (i-1) × system window height / (K-1). K is an integer greater than or equal to 2. Yi is the i-th Y-axis reference line after the K Y-axis reference lines are arranged along the Y-axis direction.

[0188] like Fig.12 As shown in (1), the height of the system window is H1. When K=2, it is determined that the system window includes 2 Y-axis reference lines (including y11 and y1k), at which time y11=0, and y1k=H1. When K=3, it is determined that the system window includes 3 Y-axis reference lines (including y11, y12, y1k), at which time y11=0, y12=H1 / 2, and y1k=H. For other values ​​of K, please refer to this description and will not be repeated here.

[0189] S202. Determine K dirty areas among the J dirty areas that are closest to the K Y-axis reference lines, wherein the i-th dirty area among the K dirty areas is closest to the i-th Y-axis reference line.

[0190] The i-th dirty area being closest to the i-th Y-axis reference line means that the distance between the center of the i-th dirty area and the i-th Y-axis reference line is smaller than the distance between the center of the dirty area other than the i-th dirty area in the user interface and the i-th Y-axis reference line.

[0191] For example, Fig.12As shown in (2), when K=2, the center distance y11 of dirty area view11 is closest to dirty area view12-dirty area view14. The center distance y1k of dirty area view14 is closest to dirty area view12-dirty area view13. When K=3, dirty area view11, dirty area view12 and dirty area view14 are determined to be the dirty areas closest to the Y-axis reference line.

[0192] S203. Merge the other dirty areas in the J dirty areas except the K dirty areas in sequence with the K dirty areas according to preset rule 1, to obtain K ROIs relative to the window in the system window.

[0193] The i-th ROI relative to the window among the K ROIs relative to the window in the system window is the ROI that includes J i The J is the smallest rectangular area among the dirty areas. i The distance between a dirty region and the i-th dirty region among the K dirty regions is closer than the distance between the dirty regions other than the i-th dirty region among the K dirty regions. That is, the number of dirty regions included in the K ROIs relative to the window is J in total.

[0194] The preset rule 1 referred to when merging the dirty areas other than the K dirty areas in the J dirty areas with the K dirty areas in sequence is: q1=1, and operation 1 is performed. Operation 1 includes: merging the q1th dirty area (referred to as dirty area q1) among the J dirty areas other than the K dirty areas with a target dirty area among the K dirty areas, and the center of the target dirty area is closest to the center of the dirty area q1 compared to the dirty areas other than the target dirty area among the K dirty areas. The dirty area q1 is merged with the target dirty area to obtain a merged dirty area, and the merged dirty area is a rectangular area with the smallest area including the dirty area q1 and the target dirty area. After the q1th merge is completed, the merged dirty area will be used as one of the K dirty areas to replace the target dirty area. Then q1=1+q1 is performed. Operation 1 is repeated until all the dirty areas other than the K dirty areas in the J dirty areas are merged with the K dirty areas, and K ROIs relative to the window in the system window can be obtained.

[0195] For example, Fig.12As shown in (3), when K is equal to 2, the K dirty areas include dirty area view11 and dirty area view14, and the target dirty area of ​​dirty area view12 (an exemplary dirty area q1) can be determined as dirty area view11. Then, the dirty area view12 and the dirty area view11 are merged to obtain a merged dirty area (denoted as R

[11] ). Subsequently, the dirty area R11 and the dirty area view14 are taken as K dirty areas. The target dirty area of ​​dirty area view13 (an exemplary dirty area q1) is determined as dirty area view14. Then, the dirty area view14 and the dirty area view13 are merged to obtain a merged dirty area (denoted as R

[12] ). After the merging is completed, the dirty area R

[11] and the dirty area R

[12] are taken as two ROIs relative to the window.

[0196] For example, Fig.12 As shown in (3), when K is equal to 3, the K dirty regions include dirty region view11, dirty region view12, and dirty region view14. The target dirty region of dirty region view13 (an exemplary dirty region q1) can be determined as dirty region view14. Then, dirty region view13 and dirty region view14 are merged to obtain a merged dirty region (denoted as R

[13] ). After the merging is completed, dirty region view11, dirty region view12, and dirty region R

[13] are used as three ROIs relative to the window.

[0197] S102b. The image mixing renderer uses the dirty area corresponding to the surface window as the ROI of the surface window relative to the window.

[0198] For an exemplary description of step S102b, please refer to the above Figure 4B as well as Figure 5A Description of the relevant content in . Figure 4B The dirty area 21 of the middle surface window 1 can be used as Figure 5A The ROI (eg, ROI21a) relative to the window in the middle surface window 1.

[0199] S103. The image mixing renderer expands Y×F+X ROIs relative to the window to obtain Y×F+X ROIs with the same width as the screen, where Y is equal to the number of system windows and X is equal to the number of surface windows.

[0200] The ROI having the same width as the screen may also be understood as the aforementioned ROI relative to the screen.

[0201] The Y system windows are system windows that request to be updated in the user interface. The X surface windows are windows that request to be updated in the user interface.

[0202] For example, Fig.13 As shown, when a system window includes two ROIs relative to the window (including ROI[11a] and ROI[12a]), ROI[11a] and ROI[12a] are expanded to obtain two ROIs with the same width as the screen, including ROI[11b] and ROI[12b].

[0203] The exemplary introduction of step S103 can also refer to the above Figure 5A as well as Figure 5B Description of the relevant content in the Fig. 7A and Figure 7B Description of the relevant content in .

[0204] In some possible cases, the image blending renderer can use an ROI (the same width as the screen) to represent the ROI (the same width as the screen) based on the position of the screen. For example, the ROI (the same width as the screen) is represented by the coordinates of the upper left corner and the lower right corner of the ROI (the same width as the screen). Among them, the coordinates of the upper left corner of the ROI (the same width as the screen) are used to indicate the coordinates of the upper left corner of the ROI (the same width as the screen) in the pixel coordinate system on the screen. The coordinates of the upper right corner of the ROI (the same width as the screen) are used to indicate the coordinates of the lower right corner of the ROI (the same width as the screen) in the pixel coordinate system on the screen. In the pixel coordinate system on the screen, the origin (0, 0) is the upper left corner of the screen, the horizontal direction is the X axis, and the vertical direction is the Y axis. The horizontal direction increases to the right and the vertical direction increases downward. The width and height of the screen are measured in pixels.

[0205] It should be understood that the left and right involved in the aforementioned coordinate system are two opposite directions, the up and down are also two opposite directions, and the horizontal direction is perpendicular to the vertical direction. This is not a specific reference and should not constitute a limitation on the embodiments of the present application.

[0206] S104. The image mixing renderer determines K11 ROIs that can be displayed in the Y×F+X areas to be updated whose width is the same as the screen.

[0207] The content in the displayable ROI will not be blocked by other content in the user interface except the content in the displayable ROI.

[0208] The width of the K11 ROIs is the same as the screen.

[0209] It should be understood that when the electronic device determines that Y×F+X areas to be updated with the same width as the screen are all displayable, the K11 ROIs are Y×F+X areas to be updated with the same width as the screen.

[0210] In some possible cases, step S104 is optional. When step S104 is not executed, the K11 ROIs that can be displayed are Y×F+X regions to be updated (ROIs) with the same width as the screen, that is, the K11 ROIs are Y×F+X regions to be updated relative to the screen.

[0211] S105a. The multi-window DPUROI calculation module in the image mixing renderer determines T1 DPUROI based on the K11 ROIs. When K11 is greater than K1, T1 is equal to K1. When K11 is less than or equal to K1, T1 is equal to K11.

[0212] When K11 is less than or equal to K1, the image mixing renderer uses K11 ROIs (with the same width as the screen) as K11 DPU ROIs. At this time, T1 is equal to K11.

[0213] When K11 is greater than K1, the image mixing renderer merges K11 ROIs (with the same width as the screen) to obtain K1 DPU ROIs in the user interface. At this time, T1 is equal to K1. K1 is an integer greater than or equal to 1.

[0214] The manner in which the multi-window DPUROI calculation module merges K11 ROIs (with the same width as the screen) into K1 DPUROI includes but is not limited to the following preset manner 12.

[0215] Fig.14 An exemplary flow chart for determining K1 DPUROIs in a user interface is shown.

[0216] Fig.15 An exemplary schematic diagram of determining K1 DPUROIs in a user interface is shown.

[0217] For the relevant contents of preset method 12, please refer to the following Fig.14 Description of steps S301-S303 in .

[0218] S301. Determine K1 Y-axis reference lines on the screen.

[0219] When K1 is greater than or equal to 2, the K1 Y-axis reference lines in the screen can divide the screen into K1-1 areas. The value of the i-th Y-axis reference line among the K1 Y-axis reference lines is yi=(i-1)×screen height / (K1-1). K1 is an integer greater than or equal to 2. Yi is the i-th Y-axis reference line after arranging the K1 Y-axis reference lines along the Y-axis direction. When K1 is equal to 1, the Y-axis reference line is 0.

[0220] like Fig.15As shown in (1), the screen height is H2. When K1=1, the screen includes 1 Y-axis reference line (including y21), and y21=0. When K1=2, it is determined that the screen includes 2 Y-axis reference lines (including y21 and y2k), and y21=0, and y2k=H2. When K1=3, it is determined that the screen includes 3 Y-axis reference lines (including y21, y22, y2k), and y21=0, y22=H / 2, and y2k=H2. For other values ​​of K, please refer to this description and will not be repeated here.

[0221] S302. Determine K1 ROIs among the K11 ROIs that are closest to the K1 Y-axis reference line, and the rth ROI among the K1 ROIs is closest to the rth Y-axis reference line.

[0222] Step S302 and the following step S303 are executed when K1 is greater than or equal to 2. When K1 is equal to 1, step S302 and the following step S303 are not executed. When K1=1, K11 ROIs are merged into one DPUROI. Fig.15 As shown in (2), the K11 ROIs include ROI211, ROI212, and ROI213. The image mixing renderer can merge ROI211, ROI212, and ROI213 into one DPUROI (eg, DPU ROI

[10] ).

[0223] When K2 is greater than or equal to 2, the description of step S302 may refer to the following content.

[0224] The rth Y-axis reference line is the rth Y-axis reference line among the K1 Y-axis reference lines.

[0225] The rth ROI is closest to the rth Y-axis reference line, which means that the distance between the center of the rth ROI and the rth Y-axis reference line is smaller than the distance between the center of ROIs other than the rth ROI among K11 ROIs and the rth Y-axis reference line.

[0226] For example, Fig.15 As shown in (2), when K1=2, the center distance y21 of ROI212 is closest to ROI212 and ROI213. The center distance y11 of ROI213 is closest to ROI212 and ROI211.

[0227] S303. Merge the other ROIs except the K1 ROI among the K11 ROIs with the K1 ROI in sequence according to preset rule 2 to obtain K1 DPU ROIs in the user interface.

[0228] In some possible cases, the rth DPU ROI among the K1 DPU ROIs in the user interface includes Gr The ROI with the smallest area relative to the screen is the rectangular area. r The distance of the ROI relative to the screen to the rth ROI among the K1 ROIs is closer than the distance to the ROIs other than the rth ROI among the K1 ROIs. That is, the total number of ROIs included in the K1 DPU ROI is K11. r ROIs relative to the screen belong to K11 ROIs.

[0229] In some possible cases, the preset rule 2 referred to when merging the other ROIs except the K1 ROI among the K11 ROIs with the K1 ROI in sequence is: q2=1, and operation 2 is performed, and operation 2 includes: merging the q2th ROI (denoted as ROIq2) among the other ROIs except the K1 ROI among the K11 ROIs with a target ROI among the K1 ROIs, and the center of the target ROI is closest to the center of ROIq2 compared to the ROIs except the target ROI among the K1 ROIs. Among them, ROIq2 and the target ROI can be merged to obtain a merged ROI, and the merged ROI is a rectangular area with the smallest area including ROIq2 and the target ROI. After the q2th merger is completed, the merged ROI will be used as one of the K1 ROIs to replace the target ROI. Then q2=1+q2 is performed. Operation 2 is repeatedly performed until all other ROIs except the K1 ROI among the K1 ROIs are merged with the K1 ROI, thereby obtaining K1 DPU ROIs on the screen.

[0230] In some other possible cases, the preset rule 2 referred to when merging the other ROIs except the K1 ROI among the K11 ROIs with the K1 ROI in sequence is: q3=1, and operation 3 is performed, and operation 3 includes: merging the q3th ROI (denoted as ROIq3) among the other ROIs except the K1 ROI among the K11 ROIs with a target ROI among the K1 ROIs, and compared with the ROIs except the target ROI among the K1 ROIs, the area after ROIq3 is merged with the target ROI is smaller than the area when ROIq3 is merged with the ROIs except the target ROI among the K1 ROIs. Among them, ROIq3 can be merged with the target ROI to obtain a merged ROI, and the merged ROI is a rectangular area with the smallest area including ROIq3 and the target ROI. After the q3th merger is completed, the merged ROI will be used as one of the K1 ROIs to replace the target ROI. Then q3=1+q3 is performed. Repeat operation 2 until all other ROIs except the K1 ROI are merged with the K1 ROI, and then K1 DPUROIs on the screen can be obtained. Fig.15 As shown in (2), the sum of the areas of ROI 212 and ROI 211 (S ROI212+ROI211 ) is smaller than the sum of the areas of ROI212 and ROI213 (S ROI212+ROI213 ), ROI211 is used as the target ROI, and ROI212 is merged with ROI211 to obtain DPU ROI

[11] . ROI213 is used as a DPU ROI (DPU ROI

[12] ). ROI212+ROI211 ) is greater than the sum of the areas of ROI212 and ROI213 (S ROI212+ROI213 ), ROI213 is used as the target ROI, and ROI212 and ROI213 are merged to obtain DPU ROI

[22] . ROI211 is used as a DPU ROI (DPU ROI

[21] ).

[0231] It should be noted here that when a region is represented by the coordinates of the upper left corner and the lower right corner of a region, Fig.16As shown in the figure, when two ROI regions (such as ret1 and ret2) are merged to obtain a new region, the coordinates of the upper left corner of the new region are [min(x_left1,x_lift2),min(y_left1,y_lift2)], and the coordinates of the lower right corner are [max(x_right1,x_right2),max(y_right1,y_right2)]. The coordinates of the upper left corner of ret1 are (x_left1,y_lift1), and the coordinates of the lower right corner are (x_right1,y_right1). The coordinates of the upper left corner of ret2 are (x_left2,y_lift2), and the coordinates of the lower right corner are (x_right2,y_right2). min means taking the minimum value, and max means taking the maximum value.

[0232] It should be understood that the design concept of step S303 and the design concept of the aforementioned step S203 can refer to each other.

[0233] In some other possible cases, the multi-window DPUROI calculation module may combine K11 ROIs (with the same width as the screen) into K1 DPUROI in addition to the aforementioned preset method 12 and also include preset method 22.

[0234] In the preset method 22, when there are X ROIs including surface windows (such as surfaceview or textureview) among the K11 ROIs, and X is less than K1, the X ROIs including surface windows can be used as X DPUROIs, that is, one ROI including a surface window among the X ROIs including surface windows can be used as a DPUROI. The remaining K1-X DPUROIs can be determined based on the other ROIs (K11-X ROIs) among the K11 ROIs except the X ROIs including surface windows in the preset method 12. The determination process of the K1-X DPUROIs includes: determining K1-X Y-axis reference lines in the screen; then determining K1-X to-be-updated regions (ROIs) in the K11-X ROIs that are closest to the K1-X Y-axis reference lines; merging the other to-be-updated regions in the K11-X ROIs except the K1-X to-be-updated regions with the K1-X to-be-updated regions in sequence according to preset rule 2 to obtain K1-X DPUROIs, so that the cth DPUROI in the K1-X DPUROIs is a rectangular region with the smallest area including Gc to-be-updated regions in the K1-X ROIs, and the distance between the Gc to-be-updated regions and the cth to-be-updated region in the K1-X to-be-updated regions is closer than the distance between the Gc to-be-updated regions and the to-be-updated regions except the cth to-be-updated region in the K1-X to-be-updated regions.

[0235] In some possible cases, the X ROIs including the surface windows should not overlap with the ROIs other than the X ROIs including the surface windows in the K11 ROIs.

[0236] In some possible cases, the control type of the surface window involved in the X ROIs including the surface window may be surfaceview or textureview.

[0237] In some other possible cases, the control type of the surface window involved in the X ROIs including the surface window may be surfaceview and the format of the content included in the surface window may be YUV, or the control type of the surface window may be textureview.

[0238] like Fig.17 As shown, here, K1 is equal to 2 or K1 is equal to 3 as an example for explanation. X is 1, that is, there is an ROI (denoted as ROI31) including a surface window among the K11 ROIs. Then ROI31 is used as a DPUROI (for example, DPU ROI

[31] ). When K1 is equal to 2, it is necessary to determine another DPUROI. Then, the aforementioned preset method 11 can be adopted to determine based on the other ROIs (including ROI32, ROI33 and ROI34) among the K11 ROIs except the X ROIs including the surface window, and obtain 1 DPUROI, for example, DPU ROI

[32] . When K1 is equal to 2, two DPUROIs need to be determined. Then, the aforementioned preset method 11 can be adopted to determine the other ROIs (including ROI32, ROI33 and ROI34) among the K11 ROIs except the X ROIs including the surface window to obtain two DPUROIs, such as DPU ROI

[41] and DPU ROI

[42] .

[0239] S105b. The multi-window DDICROI calculation module in the image mixing renderer determines T2 DDIC ROIs based on the K11 ROIs. When K11 is greater than K2, T2 is equal to K2. When K11 is less than or equal to K2, T2 is equal to K11.

[0240] K2 is an integer greater than or equal to 2.

[0241] The method of determining T2 DDICROIs in step S105b is similar to the method of determining T1 DDICROIs in step S105a. Delete the content where K1 is equal to 1, replace the multi-window DPUROI calculation module with the multi-window DDICROI calculation module, replace T1 with T2, and replace DPU ROI with DDICROI. No further details will be given here.

[0242] S106. The GPU updates the system window based on the dirty region included in the system window to obtain an updated system window, and updates the surface window to obtain an updated surface window.

[0243] The relevant contents of step S106 can refer to the above Fig. 6A Middle (1) Fig. 6A The description of (2) and related contents will not be repeated here.

[0244] S107. The DPU mixes the updated system window and the updated surface window, determines the update content in T1 DPU to-be-updated areas in the mixed window, and obtains a DPU image.

[0245] The relevant contents of step S107 can be referred to the aforementioned Fig. 6A The description of (3) and related contents will not be repeated here.

[0246] S108. DDIC determines the update content in each of the T2 DDIC to-be-updated areas in the DPU image based on the T2 DDIC ROIs to obtain a DDIC image.

[0247] For the relevant contents of step S108, please refer to the above Figure 6B The description of (1) and related contents will not be repeated here.

[0248] S109. DDIC updates the content in the area occupied by T2 DDIC ROIs in the user interface using the DDIC image to obtain an updated user interface.

[0249] The relevant contents of step S109 can be referred to in the above Figure 6B The description of (2) and related contents will not be repeated here.

[0250] In some possible cases, the aforementioned interface update method is applicable to the scenario where the screen is not forced to update. In the scenario where the screen is forced to update, the entire screen still needs to be refreshed.

[0251] It should be understood here that the aforementioned interface update method is to perform software-based calculations in the image rendering engine and the image mixing renderer to obtain DPUROI and DDICROI, and then perform user updates based on hardware such as GPU, DPU, and DDIC using DPUROI and DDICROI. In other possible cases, software-based calculations may not be performed, and the user interface update operation is performed by hardware (including hardware such as GPU, DPU, and DDIC).

[0252] Fig.18A schematic diagram showing user interface updates through hardware.

[0253] The process of updating the user interface through hardware may refer to the description of steps 1 to 4 below.

[0254] Step 1. The graphics processing unit (GPU) performs a cyclic redundancy check (CRC) on each line of data in each window of the user interface in a row unit, determines that there is a dirty area in each window with content to be updated, updates the content in the dirty area, and obtains a content dirty area in each window. A content dirty area refers to a dirty area including updated content.

[0255] The GPU transfers each window with dirty content to the display processing unit (DPU).

[0256] This step 1 corresponds to Fig.18 ① in the middle.

[0257] Step 2. The DPUROI processing module in the DPU synthesizes each window carrying the dirty content area, and then performs a CRC check on each row of data in the synthesized window to determine the DPU content dirty area whose content is to be updated. The DPU content dirty area can be understood as the updated content in the DPUROI. The DPU content dirty area is encapsulated into a data packet and sent to the display serial interface (DSI).

[0258] In some possible cases, the DPU ROI processing module can be placed in the DPU layer mixer.

[0259] This step 2 corresponds to Fig.18 ② in the middle.

[0260] Step 3. The display serial interface (DSI) in the DPU performs a CRC check on the data packet when sending the DPU content dirty area to the (display driver chip) DDIC. If the data packet including the DPU content dirty area is the same as the data packet sent to the DDIC last time, the data is not sent to the DDIC. If the data packet including the DPU content dirty area is different from the data packet sent to the DDIC last time, the DPU content dirty area is sent to the DDIC.

[0261] This step 3 corresponds to Fig.18 ③ in the middle.

[0262] Step 4. The display driver chip (DDIC) obtains the DDIC content dirty area based on the DPU content dirty area. The DDIC content dirty area can be understood as the updated content in the DDICROI.

[0263] Then, DDIC performs a CRC check on each row in the screen. If they are the same, no refresh is performed. If they are not the same, the row is refreshed based on the dirty area of ​​DDIC content.

[0264] This step 4 corresponds to Fig.18 ④ in the middle.

[0265] The following is an introduction to an exemplary electronic device provided in an embodiment of the present application.

[0266] Fig.19 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0267] The following is a detailed description of the embodiment using an electronic device as an example. It should be understood that the electronic device may have more than Fig.19 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Fig.19 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0268] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194 and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0269] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0270] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0271] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microled, micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0272] The electronic device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0273] The ISP is used to process data fed back by the camera 193 .

[0274] The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0275] Video codecs are used to compress or decompress digital videos. Electronic devices can support one or more video codecs. In this way, electronic devices can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0276] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0277] In the embodiment of the present application, the processor 110 can call the computer instructions stored in the internal memory 121 to enable the terminal to execute the interface updating method in the embodiment of the present application.

[0278] It should be understood here that the i-th content (such as dirty areas, reference lines, etc.) mentioned in this application can be the i-th content taken from top to bottom with the first direction as a reference. The same is true for other descriptions with order, for example, the r-th Y-axis reference line is the r-th Y-axis reference line from top to bottom along the first direction. Among them, i, r, etc. are used to distinguish content and do not have the meaning of quantity.

[0279] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0280] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0281] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0282] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An interface updating method, characterized in that: The method comprises: The electronic device displays a user interface on a screen; In response to a request to update the user interface, the update refers to an update of a local area in the user interface; The electronic device determines K2 first areas to be updated, wherein the K2 first areas to be updated include dirty areas in the user interface; wherein the dirty area is a rectangular area occupied by the controls to be updated in the user interface; the area of ​​the first area to be updated is smaller than the area of ​​the screen area; and K2 is an integer greater than or equal to 1; The electronic device refreshes the K2 first areas to be updated in the screen, and screen areas other than the first areas to be updated are not refreshed.

2. The method according to claim 1, characterized in that The width of the first area to be updated is the same as the width of the screen.

3. The method according to claim 1 or 2, characterized in that: The K2 is equal to 1, and the first area to be updated is a rectangular area with the smallest area including all dirty areas in the user interface.

4. The method according to any one of claims 1 to 3, characterized in that The K2 is greater than or equal to 2, the K2 first areas to be updated do not overlap, the width of the first area to be updated is the same as the width of the screen, and the K2 first areas to be updated are arranged in sequence along the first direction, and the K2 first areas to be updated jointly cover the dirty areas in the user interface; in the first direction, the upper edge of the topmost first area to be updated is above the upper edge of the topmost dirty area, and the lower edge of the bottommost first area to be updated is below the lower edge of the bottommost dirty area. The first direction is the direction of line-by-line scanning from top to bottom for screen display.

5. The method according to claim 4, characterized in that The dirty areas in the user interface are arranged sequentially along a first direction, and the middle dirty areas except the topmost dirty area and the bottommost dirty area are covered by the area to be updated closest to the middle dirty area, and the covering means that the lower edge of the closest area to be updated is below the lower edge of the middle dirty area or coincides with the lower edge of the middle dirty area, and the upper edge of the closest area to be updated is above the upper edge of the middle dirty area or coincides with the upper edge of the middle dirty area.

6. The method according to claim 5, characterized in that The area to be updated that is closest to the middle dirty area includes: the area to be updated that is closest to the center point of the middle dirty area.

7. The method according to claim 6, characterized in that The area to be updated that is closest to the middle dirty area includes: the area to be updated with the smallest area after covering the middle dirty area.

8. The method according to any one of claims 1 to 7, characterized in that The K2 is greater than or equal to 2, and when the K2 first areas to be updated are arranged sequentially along the first direction, the r1th first area to be updated includes Gr1 extended dirty areas, the width of the extended dirty area is the same as the width of the first area to be updated, and the extended dirty area includes one dirty area, and when the one dirty area does not overlap with dirty areas other than the one dirty area in the user interface, the height of the extended dirty area is the same as the height of the one dirty area, or when the extended dirty area includes at least two overlapping dirty areas, the height of the extended dirty area is the same as the height of the two overlapping dirty areas after overlapping; the Gr1 is greater than or equal to 1, and when the Gr1 is greater than 1, the Gr1 extended dirty areas do not overlap with each other along the first direction, and the first direction is the direction of progressive scanning from top to bottom of the screen display.

9. The method according to claim 8, characterized in that The distance between the Gr1 extended dirty area and the r1th reference line among the K2 reference lines is closer to or equal to the distance between the K2 reference lines other than the r1th reference line; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; the r1th reference line is the r1th reference line after the K2 reference lines are arranged sequentially along the first direction.

10. The method according to claim 8, characterized in that The Gr1 extended dirty areas include a first extended dirty area, and a distance between the first extended dirty area and the r1th reference line among the K2 reference lines is closer to or equal to a distance between an extended dirty area other than the first extended dirty area in the user interface and the r1th reference line; when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are combined with the first extended dirty area to obtain the r1th first area to be updated, an area of ​​the K2 first areas to be updated is smaller than a first area, and the first area is an area of ​​the K2 first areas to be updated obtained when the extended dirty areas other than the first extended dirty area among the Gr1 extended dirty areas are combined with the second extended dirty area; a distance between the second extended dirty area and the r2th reference line among the K2 reference lines is closer to the r2th reference line between the extended dirty areas other than the second extended dirty area in the user interface; wherein the K2 reference lines are lines that divide the screen into K2-1 areas and are perpendicular to the first direction; and the r1th reference line is the r1th reference line after the K2 reference lines are arranged in sequence along the first direction.

11. The method according to any one of claims 1 to 7, characterized in that: The K2 is greater than or equal to 2, the K2 first areas to be updated are arranged in sequence along the first direction, among the K2 first areas to be updated, there are X0 first areas to be updated that are areas including surface windows, and the height of one of the X0 first areas to be updated is the same as the height of the surface window included in the area to be updated; wherein, X0 is less than the K2; the surface window is a drawable control, and the surface window does not include a drawable control; an area obtained by extending a surface window in the X0 surface windows to the same width as the first area to be updated does not include any area other than the one surface window The dirty area outside the window, the surface window does not overlap with the dirty area outside the surface window; the K2 first areas to be updated include K2-X0 first areas to be updated in addition to X0 first areas to be updated; the K0 first areas to be updated divide the user interface into K0+1 areas excluding the K0 first areas to be updated, and when the K0+1 area is greater than or equal to K2-X0, at most K2-X0 areas among the K0+1 areas include dirty areas, and the K0+1 areas except the K2-X0 areas do not include dirty areas.

12. The method according to any one of claims 1 to 11, characterized in that The user interface includes a system window for requesting an update, and the system window includes at least one drawable control; before the electronic device merges the dirty areas in the user interface into K2 non-overlapping first areas to be updated, the method further includes: The electronic device uses the rectangular area occupied by the control whose content is to be updated in the system window as a dirty area.

13. The method according to claim 12, characterized in that The user interface also includes a surface window for requesting update, the surface window is a drawable control, and the surface window does not include a drawable control; before the electronic device merges the dirty areas in the user interface into K2 non-overlapping first areas to be updated, the method also includes: The electronic device regards the surface window as a control whose content is to be updated, and regards the rectangular area occupied by the surface window as a dirty area.

14. The method according to claim 12, characterized in that The control type of the surface window is surfaceview or textureview.

15. The method according to any one of claims 1 to 14, characterized in that Before the electronic device refreshes the K2 first areas to be updated in the screen, the method further includes: the electronic device acquires updated content in the K2 first areas to be updated; The electronic device refreshing the K2 first to-be-updated areas in the screen specifically includes: The electronic device refreshes the K2 first areas to be updated in the screen based on the updated content, and the updated content is displayed in the refreshed K2 first areas to be updated.

16. The method according to claim 15, characterized in that The electronic device obtains updated content in K2 first areas to be updated, specifically comprising: the electronic device merges the dirty areas in the user interface into K1 second areas to be updated; wherein K1 is an integer greater than or equal to 1, and when K1 is greater than or equal to 2, the K1 second areas to be updated do not overlap; the content in the K1 second areas to be updated includes the content in the K2 areas to be updated; The electronic device updates the content of the K1 second areas to be updated to obtain updated content in the K1 second areas to be updated; The electronic device obtains updated content of the K2 first areas to be updated from the updated content of the K1 second areas to be updated.

17. The method according to any one of claims 12 to 14, characterized in that: The electronic device determines K2 first areas to be updated, specifically including: The electronic device merges the dirty areas in the system window into K dirty areas; K is the maximum value of K1 and K2; K1 is the number of areas to be updated supported by the display processing unit of the electronic device, and K2 is the number of areas to be updated supported by the display driver chip of the electronic device; The electronic device performs width expansion based on the K dirty regions to obtain K third regions to be updated, each of which has a width equal to that of the screen; The electronic device obtains, based on the surface window, a fourth area to be updated whose width is the same as that of the screen, and a height of the fourth area to be updated is the same as that of the surface window; The electronic device merges Y×K third areas to be updated and X fourth areas to be updated into the K2 first areas to be updated, wherein Y is the number of the system windows, and X is the number of the surface windows.

18. The method according to claim 17, characterized in that The electronic device combines Y×K third areas to be updated and X fourth areas to be updated into the K2 first areas to be updated, specifically including: The electronic device determines K2 Y-axis reference lines on the screen; The electronic device determines K2 to-be-updated areas among the Y×K third to-be-updated areas and the X fourth to-be-updated areas which are closest to the K2 Y-axis reference lines; The electronic device merges the Y×K third areas to be updated and the areas to be updated other than the K2 areas to be updated among the X fourth areas to be updated with the K2 areas to be updated in sequence according to a preset rule to obtain the K2 first areas to be updated, so that the rth first area to be updated among the K2 first areas to be updated is a rectangular area with the smallest area including the Gr areas to be updated, the distance between the Gr and the rth area to be updated among the K2 areas to be updated is closer than the distance between the Gr and the areas to be updated among the K2 areas to be updated, and the areas to be updated among the Gr areas to be updated are the third areas to be updated or the fourth areas to be updated.

19. The method according to claim 17, characterized in that The electronic device combines Y×K third areas to be updated and X fourth areas to be updated into the K2 first areas to be updated, specifically including: When it is determined that the X fourth areas to be updated do not overlap with the Y×K third areas to be updated, X is less than K2, and the X fourth areas to be updated divide the user interface into K+1 areas excluding the X fourth areas to be updated, the K+1 areas are greater than or equal to K2-X, at most K2-X areas among the K+1 areas include the third area to be updated, and areas other than the K2-X areas among the K+1 areas do not include a dirty area, the electronic device uses the X fourth areas to be updated as the X first areas to be updated; When the Y×K third areas to be updated are greater than or equal to K2-X, the electronic device determines K2-X Y-axis reference lines in the screen; The electronic device determines K2-X to-be-updated areas which are closest to the K2-X Y-axis reference lines among the Y×K third to-be-updated areas; The electronic device merges the Y×K third areas to be updated except K2-X areas to be updated with the K2-X areas to be updated in sequence according to a preset rule to obtain K2-K first areas to be updated, so that the cth first area to be updated among the K2-X first areas to be updated is the rectangular area with the smallest area including the Gc third areas to be updated, and the distance between the Gc third areas to be updated and the cth area to be updated among the K2-X areas to be updated is closer than the distance between the Gc third areas to be updated and the areas to be updated among the K2-X areas to be updated.

20. The method according to any one of claims 17 to 19, characterized in that Before the electronic device combines the Y×K third areas to be updated and the X fourth areas to be updated into the K2 first areas to be updated, the method further includes: The electronic device determines that the Y×K third areas to be updated and the X fourth areas to be updated are displayable in the updated user interface.

21. An electronic device, characterized in that: include: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-20.

22. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 20.

Images