Popup window adjustment method, graphical interface and related device

Abstract

The invention discloses a pop-up window adjusting method, a graphical interface and a related device.In the method, electronic equipment can adjust a pop-up window under the condition that a part of content of the pop-up window exceeds a display area of a display screen, the adjustment comprises the steps that the pop-up window is shrunk, or interface elements are newly added in the display area, and the pop-up window is displayed on the graphical interface. The newly added interface element and the interface element exceeding the display area in the popup window are used for triggering the same function, or the interface element exceeding the display area is moved to the display area. Therefore, the electronic equipment can be prevented from displaying an incomplete pop-up window, a user can read contents in the pop-up window conveniently, or the user can control the pop-up window conveniently, and the experience feeling of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a pop-up window adjustment method, a graphical interface and related devices. Background Art

[0002] A pop-up window refers to a window that pops up during the running of an application, which is used to show users prompt information other than the application page.

[0003] Although pop-up windows are widely used in the terminal field, the pop-up windows of some applications cannot be well adapted to electronic devices, resulting in part of the content of the pop-up window exceeding the display area of ​​the display screen, and users cannot view the complete content of the pop-up window. In addition, if the content exceeding the display area includes interactive controls, such as close controls, confirmation controls, etc., users may not be able to operate the pop-up window normally, affecting the user experience. Summary of the invention

[0004] The present application provides a pop-up window adjustment method, a graphical interface and related devices, which realize the adjustment of inappropriate pop-up windows, making the adjusted pop-up windows more suitable for user browsing and control, and improving the user experience.

[0005] In a first aspect, an embodiment of the present application provides a pop-up window adjustment method, the method comprising: an electronic device displays a first pop-up window, and a first interface element of the first pop-up window exceeds a display area of ​​the display screen; the electronic device adjusts the first pop-up window, and the adjustment comprises: shrinking the first pop-up window, or displaying a second interface element in the display area, the second interface element and the first interface element are used to trigger the same function, or moving the first interface element into the display area.

[0006] By implementing the method provided by the first method, the electronic device can adjust the pop-up window when displaying an incomplete pop-up window so that the adjusted pop-up window can display complete content, making it easier for users to manipulate the pop-up window or view the complete content in the pop-up window, thereby improving the user experience.

[0007] In combination with the first aspect, in a possible implementation, when the electronic device displays the first pop-up window, the height of the first interface element is 0, and before shrinking the first pop-up window, the method also includes: increasing the height of the first interface element.

[0008] Among them, increasing the height of the interface element before shrinking the pop-up window can ensure that the first interface element can be displayed in the user interface after the pop-up window is shrunk, ensuring that the electronic device 100 can adjust the content of the pop-up window located outside the display screen to within the range of the display screen.

[0009] In a second aspect, an embodiment of the present application provides a pop-up window adjustment method, the method comprising: the electronic device displays a first pop-up window, the size of the first pop-up window is greater than a first threshold; the electronic device adjusts the first pop-up window, the adjustment comprising: reducing the first pop-up window.

[0010] By implementing the method provided in the second aspect, the electronic device can reduce the size of the pop-up window when the size of the pop-up window is too large, so that the electronic device can display a pop-up window of appropriate size, which is convenient for users to read the content in the pop-up window. At the same time, it also avoids the situation where the price control part in the pop-up window is too dispersed due to the pop-up window being too large, affecting the user's control over the pop-up window.

[0011] In combination with the first aspect and the second aspect, in a possible implementation method, the first pop-up window is displayed in the first user interface. Before the electronic device adjusts the first pop-up window, the method also includes: the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface.

[0012] That is to say, the electronic device 100 can evaluate the information of each interface element in the user interface to find the first pop-up window from the user interface, and then successfully adjust the pop-up window.

[0013] In combination with the first aspect and the second aspect, in a possible implementation method, when the electronic device displays a first pop-up window, the number of interface elements whose layout has changed in the first user interface is greater than a second threshold, and the interface elements in the first user interface that belong to the first pop-up window are determined, specifically including: when the number of interface elements whose layout has changed in the first user interface of the electronic device is greater than the second threshold, determining the interface elements in the first user interface that belong to the first pop-up window.

[0014] In this way, the electronic device 100 can monitor the layout changes of the interface elements contained in the user interface in real time, and search for pop-up windows that need to be adjusted in the user interface based on the layout changes. This can avoid the electronic device 100 being in a state of searching for pop-up windows all the time. The characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application are fully considered. When it is determined that a pop-up window may appear in the user interface, a search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0015] In combination with the first aspect and the second aspect, in a possible implementation manner, the electronic device determines the interface elements belonging to the first pop-up window in the first user interface within a preset time period.

[0016] In this way, it is possible to avoid the electronic device 100 spending too much time searching for pop-up windows, and control the power consumption of the electronic device 100.

[0017] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface, specifically including: the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward; the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

[0018] That is to say, when traversing the interface elements, the electronic device 100 can traverse from the bottom layer of the ViewTree upwards. This is because if the user interface includes pop-up windows, in the ViewTree, the interface elements that are closer to the lower layer are drawn later and are more likely to be pop-up windows. Therefore, traversing from the bottom layer of the ViewTree upwards can minimize the number of interface elements traversed by the electronic device 100. When there are pop-up windows in the user interface, the interface elements that meet the requirements can be found as quickly as possible, thereby improving the efficiency of the electronic device 100 in finding pop-up windows.

[0019] In combination with the first aspect and the second aspect, in a possible implementation, after the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface, the method also includes: the electronic device stops traversing after traversing to an interface element whose size is greater than a third threshold.

[0020] That is to say, when the electronic device 100 is traversing the interface elements, if it finds an interface element of a larger size, it can stop traversing. This is because if the user interface contains a pop-up window, when the electronic device 100 traverses to an interface element of a larger size, it means that most or all of the interface elements in the pop-up window may have been traversed. The electronic device 100 can end the traversal to avoid introducing interface elements that do not belong to the pop-up window into the calculation process during the search process, thereby saving search time.

[0021] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device is pre-installed with a whitelist, and the whitelist includes the IDs of the interface elements contained in one or more pop-up windows; the electronic device determines the interface elements in the first user interface that belong to the first pop-up window based on one or more information of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface, specifically including: the electronic device searches for a third interface element in the first user interface that is in the whitelist; the electronic device determines the interface element in the first user interface that belongs to the first pop-up window through the third interface element.

[0022] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains an interface element in the whitelist.

[0023] In combination with the first aspect and the second aspect, in a possible implementation, the third interface element is an interface element corresponding to the background contained in the first pop-up window, and the interface element included in the first pop-up window is a sub-element of the third interface element.

[0024] In combination with the first aspect and the second aspect, in a possible implementation, the third interface element is the element with the highest level in the view tree ViewTree of the first user interface among all the interface elements included in the first pop-up window.

[0025] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on one or more information of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, specifically including: the electronic device searches for the specified interface element based on multiple information of the interface elements contained in the first user interface; the specified interface element is the interface element contained in the pop-up window; the electronic device determines the interface element belonging to the first pop-up window in the first user interface through the specified interface element.

[0026] That is, the electronic device 100 may first search for a landmark element in the pop-up window and then use the landmark element to search for the pop-up window, so that the electronic device 100 can quickly find the pop-up window.

[0027] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device searches for a specified interface element based on multiple information of the interface elements contained in the first user interface, specifically including: the electronic device determines the scores of multiple interface elements in the first user interface based on the difference between the multiple information of the interface elements contained in the first user interface and the multiple information of the specified interface element, the score of an interface element indicating the possibility that the interface element is the specified interface element; the electronic device searches for the specified interface element from the multiple interface elements based on the scores of the multiple interface elements.

[0028] Among them, the electronic device 100 calculates the score of the interface element through multiple information of the interface element, and can combine multiple factors to evaluate the similarity between the found interface element and the specified interface element from multiple aspects, thereby improving the accuracy and comprehensiveness of searching for the specified interface element.

[0029] In combination with the first aspect and the second aspect, in a possible implementation method, the designated interface element is any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before the electronic device determines the scores of multiple interface elements in the first user interface, the method also includes: the electronic device filters out multiple interface elements that contain interactive events from the interface elements in the first user interface.

[0030] In this way, when calculating the scores of interface elements, the electronic device 100 can only calculate the scores of the multiple screened interface elements and search for the specified interface element from these screened interface elements, thereby reducing the amount of calculation of the electronic device 100 and speeding up the search speed of the electronic device 100.

[0031] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface by specifying the interface element, specifically including: in the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, the scrolling page layout, and the Tab page layout, the electronic device obtains a first element group, and the first element group includes: all child elements of the parent element of the interface element whose size is greater than a third threshold that the electronic device traverses upward from the bottom layer of the view tree ViewTree of the first user interface; when the first element group meets the layout conditions of the first pop-up window, the electronic device determines the first element group as the interface element belonging to the first pop-up window in the first user interface.

[0032] Among them, evaluating the page layout of the parent element of the specified interface element can further determine whether the specified interface element searched by the electronic device 100 is the interface element in the pop-up window, thereby improving the accuracy of the pop-up window search.

[0033] In combination with the first aspect and the second aspect, in a possible implementation method, whether the first element group meets the layout conditions of the first pop-up window is determined by the electronic device based on the score of the first element group calculated based on the position, type, interaction event, width, and height of the interface elements contained in the first pop-up window, wherein the score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

[0034] It can be seen that by using multiple information of interface elements in the first pop-up window to calculate the score of the first element group, multiple factors can be used to evaluate the similarity between the first element group and the pop-up window from multiple aspects, thereby improving the accuracy and comprehensiveness of pop-up window search.

[0035] In combination with the first aspect and the second aspect, in a possible implementation, the display screen is a folding screen, and when the electronic device displays the first pop-up window, the folding screen is in an unfolded state.

[0036] In other words, this method can address the problem that the pop-up window displayed when the folding screen is in the unfolded state exceeds the display area of ​​the display screen, and adjust the pop-up window so that the adjusted pop-up window is easier for users to read and operate, thereby improving the user experience. Alternatively, this method can address the problem that the pop-up window displayed when the folding screen is in the unfolded state is too large, and then reduce the pop-up window to facilitate users to read the content in the pop-up window. At the same time, it also avoids the problem that the pop-up window is too large, which makes the controls in the pop-up window too scattered, affecting the user's control of the pop-up window, thereby improving the user experience.

[0037] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method described in the first aspect or any possible implementation manner of the first aspect, the second aspect or any possible implementation manner of the second aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, cause the electronic device to execute a method as described in the first aspect or any possible implementation of the first aspect, or the second aspect or any possible implementation of the second aspect.

[0039] In a fifth aspect, a computer program product, when the computer program product runs on a computer, enables the computer to execute the method described in the first aspect or any possible implementation of the first aspect, the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1A-1E , Figure 2A-2E Some user interfaces provided for embodiments of the present application;

[0041] Figure 3 A schematic diagram of a flow chart of monitoring the layout of a user interface by an electronic device 100 provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a ViewTree provided in an embodiment of the present application;

[0043] Figure 5 To draw a flowchart of interface elements;

[0044] Figure 6 An overall flow chart for searching and adjusting pop-up windows provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a pop-up window in ViewTree provided in an embodiment of the present application;

[0046] Figure 8 A flowchart of an overall method for an electronic device 100 to search for a pop-up window through a specified interface element contained in the pop-up window provided in an embodiment of the present application;

[0047] Fig. 9 A detailed flow chart of the electronic device 100 provided in the embodiment of the present application screening out interface elements that meet the requirements from interface elements of the user interface;

[0048] Fig.10 A schematic diagram of a process for searching for a specified interface element provided in an embodiment of the present application;

[0049] Fig.11 A schematic diagram of dividing a display screen into nine areas provided in an embodiment of the present application;

[0050] Fig.12 A schematic diagram of a method flow for searching a pop-up window according to a specified interface element provided in an embodiment of the present application;

[0051] Fig.13 A schematic diagram of a process for reducing a pop-up window provided in an embodiment of the present application;

[0052] Fig.14 A schematic diagram of a process of adjusting the height of a specified interface element and reducing a pop-up window by the electronic device 100 provided in an embodiment of the present application;

[0053] Fig.15 A flowchart of a pop-up window adjustment method provided in an embodiment of the present application;

[0054] Fig.16 A flowchart of another pop-up window adjustment method provided in an embodiment of the present application;

[0055] Fig.17 A schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0056] Fig.18 A schematic diagram of the device form of the electronic device 100 provided in an embodiment of the present application;

[0057] Fig.19 A software structure block diagram of the electronic device 100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0059] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0060] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0061] Pop-ups are a way for users to communicate with application products. They can be used to convey information, provide status feedback, guide users, and so on.

[0062] However, the pop-up windows of some applications cannot be well adapted to electronic devices. Commonly, there is a problem that pop-up windows cannot be properly adapted on folding screen devices. Since the folding screen includes a folded state and an unfolded state, and the display area in the folded state is smaller than the display area in the unfolded state, if the application has a pop-up window that needs to be displayed, when the folding screen is in the folded state, a normal-sized pop-up window can be displayed. However, when the folding screen is in the unfolded state, since the display area in the unfolded state is larger than the display area in the folded state, the application will generally automatically enlarge the pop-up window that needs to be displayed in proportion according to the size of the display area, resulting in the device displaying a pop-up window that is too large, blocking too much content of the application page, affecting the user's perception. Furthermore, the enlarged pop-up window content is easy to exceed the display area, which not only affects the appearance of the interface, but more importantly, if the interactive controls in the pop-up window exceed the boundary of the display screen or are far away from the center of the display screen, it also affects the user's control of the pop-up window.

[0063] For details about the pop-up windows displayed in the folded state of foldable screen devices and the incompatible pop-up windows displayed in the unfolded state, please refer to the subsequent Figure 1A-1B , Figure 2A-2B content.

[0064] In order to prevent electronic devices from displaying inappropriate pop-up windows, one solution is to use a third-party application to automatically skip the pop-up windows of the application. Such a third-party application can automatically simulate a click operation on the pop-up window, triggering the electronic device to automatically skip the pop-up window that pops up in the application.

[0065] However, this has certain drawbacks:

[0066] 1) Users need to install third-party applications on their devices and authorize accessibility permissions, which makes the security of the device unprotected;

[0067] 2) When using the third-party application to skip pop-up windows, some pop-up windows require manual configuration by the user, which increases the complexity of user operations;

[0068] 3) Third-party applications control the device to automatically skip pop-up windows, so users no longer perceive the existence of pop-up windows, which may cause users to miss important information.

[0069] An embodiment of the present application provides a pop-up window adjustment method, in which an electronic device can display a pop-up window. When the pop-up window is not suitable, the electronic device can adjust the pop-up window, and the adjustment includes any one of the following: reducing the pop-up window, or adding an interface element, the newly added interface element and the interface element that exceeds the display area in the pop-up window are used to trigger the same function, or moving the interface element that exceeds the display area into the display area.

[0070] In some embodiments, the pop-up window may not be compatible when part of the displayed content of the pop-up window exceeds the display area of ​​the display screen, for example, part of the displayed content may include the first interface element of the first pop-up window. The content exceeding the display area in the pop-up window may be information such as text or pictures in the pop-up window, which makes it impossible for the user to view the complete prompt information displayed to the user in the pop-up window, or the content exceeding the display area in the pop-up window may refer to interactive controls in the pop-up window, such as close controls, confirmation controls, etc., which makes it impossible for the user to normally operate the pop-up window, affecting the user's experience.

[0071] In some implementations, the pop-up window may not fit properly because the size of the pop-up window is too large. If the size of the pop-up window is too large, the pop-up window will block most of the content of the application page, affecting the browsing effect of the user.

[0072] It can be seen that when the pop-up window displayed on the electronic device is not suitable, the pop-up window adjustment method provided in the embodiment of the present application can reduce the pop-up window so that the electronic device can display a pop-up window of appropriate size. Alternatively, a new interface element for controlling the pop-up window can be added to the display area, or the interface elements in the pop-up window that are outside the display area can be moved into the display area, so that the user can control the pop-up window or view the complete content in the pop-up window, and read the content in the pop-up window. At the same time, it also avoids the situation where the pop-up window is too large, which makes the controls in the pop-up window too scattered, affecting the user's control of the pop-up window.

[0073] The following takes the electronic device 100 as a folding screen device as an example. Figure 1A-1E , Figure 2A-2E Some user interfaces provided by the embodiments of the present application are exemplarily introduced.

[0074] Figure 1A and Figure 1B The user interfaces 10 are respectively displayed by the electronic device 100 in the folded state and the unfolded state, and the user interface 10 may be the user interface of application A. The application A may refer to any application installed in the electronic device 100 that has a pop-up window during the display process. The embodiment of the present application does not limit the application A. The user interface 10 may include a home page 101 and a pop-up window 102 provided by the application A. The home page 101 may be used to display multiple functions provided by the application A, and the pop-up window 102 may be used to display some prompt information of the application.

[0075] It is understandable that the pop-up window 102 can be a window that pops up automatically when the electronic device 100 is displaying the home page 101, or it can be a window that is displayed together with the home page 101 when the electronic device 100 is opened or switched to the home page 101.

[0076] If the electronic device 100 is in a folded state, Figure 1AAs shown, the pop-up window 102 is used to prompt the user that there is a new version of the application A. The pop-up window 102 may include: a pop-up window body 102A and a close control 102B. The pop-up window body 102A may be used to display the main content of the pop-up window 102, including text, pictures and other information. For example, in Figure 1A In the pop-up window 102 shown, the pop-up window main body 102A may include text information "New version first experience" etc. The close control 102B may be used to close the pop-up window 102. Exemplarily, when the electronic device 100 detects a user operation acting on the close control 102B, such as a click operation, the electronic device 100 may close the pop-up window 102 in response to the operation.

[0077] If the electronic device 100 displays the pop-up window 102 in the expanded state, since the display area of ​​the electronic device 100 in the expanded state is larger than the display area in the folded state, the electronic device 100 will determine the size of the pop-up window 102 according to the size of the display area when displaying the pop-up window 102. For example, the electronic device 100 can display the following in the expanded state: Figure 1B Pop-up window 102 is shown.

[0078] contrast Figure 1A and Figure 1B It can be seen that due to Figure 1B In the electronic device 100, when it is in the unfolded state, its display area is larger than Figure 1A In the embodiment, the display area of ​​the electronic device 100 is in the folded state, so Figure 1B The size of the pop-up window 102 is shown to be larger than Figure 1A Pop-up window 102 is shown.

[0079] In a specific implementation, since the width of the display area of ​​the electronic device 100 in the unfolded state is equivalent to twice the width of the display area of ​​the electronic device 100 in the folded state, the electronic device 100 in the unfolded state (see Figure 1B ) displays the pop-up window 102 in the folded state (see Figure 1A ) displays a pop-up window 102 that is proportionally enlarged by twice.

[0080] Also, see Figure 1B ,because Figure 1B The size of the pop-up window 102 shown is too large, and a portion of the content of the pop-up window 102, namely the close control 102B, exceeds the range of the display area.

[0081] In this case, the user cannot quickly close the pop-up window 102 by closing the control 102B. In order to close the pop-up window 102, the user may need to switch the electronic device 100 from the unfolded state to the folded state so that the pop-up window 102 is switched to the collapsed state. Figure 1AIn the pop-up window 102 shown, after the close control 102B of the pop-up window 102 is displayed in the display area, the user can close the pop-up window 102 through the close control 102B. However, this requires the user to change the folding form of the electronic device 100, making the user operation cumbersome and reducing the user experience.

[0082] It can be understood that compared to the folded state, in the expanded user interface 10, in addition to the larger size of the pop-up window 102, the size of each interface element contained in the home page 101, or the interval between the interface elements, can also change, so that the size of the home page 101 can adapt to the larger display area.

[0083] The pop-up window adjustment method provided in the embodiment of the present application can find the pop-up window that needs to be adjusted in the user interface, that is, the unsuitable pop-up window, and adjust the pop-up window. Figure 1B Taking the pop-up window 102 as an example, the adjustment may include:

[0084] 1) Reduce the size of the pop-up window

[0085] For example, the reduced pop-up window may refer to Figure 1C Pop-up window 102 is shown.

[0086] Combination Figure 1B and Figure 1C It can be seen that when the electronic device 100 displays Figure 1B After the user interface 10 shown in FIG. 1 is displayed, if the size of the pop-up window 102 included in the user interface 10 is too large, or the content in the pop-up window 102 exceeds the display area, the electronic device 100 can reduce the pop-up window 102 and display Figure 1C The user interface 10 shown in the figure allows the user to view all the contents in the pop-up window 102, and the user can also Figure 1C The close control 102B visible in the figure quickly closes the pop-up window 102.

[0087] 2) Add mapping interface elements

[0088] In the case where the interface elements contained in the pop-up window exceed the display area, making it impossible for the user to manipulate the pop-up window through the interface elements, a new interface element can be added to the display area, and the interactive events of the interface elements in the pop-up window that exceed the display area can be mapped to the newly added interface element. For example, if the interface element in the pop-up window that exceeds the display area is a close control, the close control may include an interactive event, which can be used to trigger the closing of the pop-up window based on the user's click operation. Then when adjusting the pop-up window, a new control can be added to the pop-up window, and the click event of the original close control can be mapped to the newly added control, so that the user can trigger the closing of the pop-up window by initiating a click operation on the newly added control.

[0089] That is, adding a mapping interface element refers to adding a new interface element to the interface. The new interface element and the interface element in the pop-up window that exceeds the display area can have the same function, so that the user can control the pop-up window through the new interface element.

[0090] For example, the newly added interface element may refer to Figure 1D The close control 102C shown. The close control 102C can be used to close the pop-up window 102.

[0091] Combination Figure 1B and Figure 1D It can be seen that when the electronic device 100 displays Figure 1B After the user interface 10 shown, if the pop-up window 102 included in the user interface 10 is too large, so that the close control 102B exceeds the display area, the electronic device 100 can add a close control 102C within the visible range of the pop-up window 102. In this way, the user can quickly close the pop-up window 102 through the close control 102C.

[0092] It is understandable that the electronic device 100 can place the newly added interface element in a blank area of ​​the user interface to prevent the newly added interface element from blocking important information in the user interface, or, further, the electronic device 100 can place the newly added interface element in a visible blank area included in the pop-up window. The embodiment of the present application does not limit the display position of the newly added interface element.

[0093] 3) Adjust the position of interface elements

[0094] If the interface elements contained in the pop-up window exceed the display area, the interface elements exceeding the display area can be moved into the display area so that the user can view the complete content of the pop-up window.

[0095] For example, see Figure 1E , the pop-up window 102 displays a close control 102B after the position is adjusted.

[0096] Combination Figure 1B and Figure 1E It can be seen that when the electronic device 100 displays Figure 1B After the user interface 10 shown, if the close control 102B in the pop-up window 102 exceeds the display area, the electronic device 100 can move the close control 102B upward and move it to the pop-up window body 102A of the pop-up window 102, so that the user can view the complete content of the pop-up window, and the user can also close the pop-up window 102 through the moved close control 102B.

[0097] It is understandable that the electronic device 100 can move the interface elements in the pop-up window that exceed the display area to the blank area of ​​the user interface to avoid blocking important information in the user interface, or, further, the electronic device 100 can move the interface elements in the pop-up window that exceed the display area to the blank area visible in the pop-up window. The embodiment of the present application does not limit the display position of the interface element after adjustment.

[0098] under Figure 2A-2E The user interface involved in adjusting another form of pop-up window according to an embodiment of the present application is shown.

[0099] Figure 2A and Figure 2B The user interfaces 20 are displayed in the folded state and the unfolded state of the electronic device 100, respectively. The user interface 20 may be a user interface of an application A installed in the electronic device 100. For a detailed description of the user interface 20, please refer to the aforementioned Figure 1A The contents in will not be repeated here.

[0100] If the electronic device 100 is in a folded state, Figure 2A As shown, the pop-up window 202 is used to prompt the user that the application applies to obtain the location information of the device. The pop-up window 202 may include: text information 202A, a rejection control 202B, and an allow control 202C. The text information 202A includes: "The application applies to obtain your location information", the rejection control 202B is used to trigger the rejection of granting the application the permission to obtain the location information of the electronic device 100, and the allow control 202C is used to trigger the granting of the application the permission to obtain the location information of the electronic device 100.

[0101] If the electronic device 100 displays the pop-up window 202 in the expanded state, since the display area of ​​the electronic device 100 in the expanded state is larger than the display area in the folded state, the electronic device 100 will determine the size of the pop-up window 202 according to the size of the display area when displaying the pop-up window 202. For example, the electronic device 100 can display the following in the expanded state: Figure 2B The pop-up window 202 shown may refer to Figure 2A The pop-up window 202 shown is a pop-up window that is enlarged twice.

[0102] Also, see Figure 2B ,because Figure 2B The size of the pop-up window 202 is too large, and part of the content of the pop-up window 202, including the text information 202A and part of the permission control 202C, exceeds the range of the display area.

[0103] In this case, the user cannot view the complete text information in the pop-up window 202 , nor can the user allow the application to obtain the location information through the permission control 202C, so that the user's operation on the pop-up window 202 is limited.

[0104] Similar to the above Figure 1A-1E The pop-up window adjustment method provided in the embodiment of the present application can be used for Figure 2B The pop-up window 202 shown is adjusted. Exemplarily, the adjustment may include:

[0105] 1) Reduce the size of the pop-up window

[0106] For example, the reduced pop-up window may refer to Figure 2C Pop-up window 202 is shown.

[0107] Combination Figure 2B and Figure 2C It can be seen that when the electronic device 100 displays Figure 2B After the user interface 20 shown in FIG. 1 is displayed, if the size of the pop-up window 202 included in the user interface 10 is too large, or part of the content of the pop-up window 202 exceeds the display area, the electronic device 100 can reduce the pop-up window 202 and display Figure 2C The user interface 20 shown in the figure allows the user to view all the contents in the pop-up window 202, and the user can also Figure 2C The permission control 202C visible in the figure grants the application permission to obtain the location information of the electronic device 100.

[0108] 2) Add mapping interface elements

[0109] The electronic device 100 can add a new interface element and map the interaction events of the interface elements in the pop-up window that exceed the display area to the newly added interface element.

[0110] For example, the newly added interface element may refer to Figure 2D The allow control 202D shown. The allow control 202D is Figure 2A The permission control 202C has the same function as that in FIG. 1 , and is used to grant the application permission to obtain the location information of the electronic device 100.

[0111] Combination Figure 2B and Figure 2D It can be seen that when the electronic device 100 displays Figure 2B After the user interface 20 shown, if the pop-up window 102 included in the user interface 20 is too large, so that the permission control 202C exceeds the display area, the electronic device 100 can add a permission control 202D within the visible range of the pop-up window 202. In this way, the user can grant the application the permission to obtain the location information of the electronic device 100 through the permission control 202D.

[0112] 3) Adjust the position of interface elements

[0113] For example, see Figure 2E The pop-up window 102 displays the text information 202A after the position adjustment, the rejection control 202B, and the permission control 202C.

[0114] Combination Figure 2B and Figure 2E It can be seen that when the electronic device 100 displays Figure 2B After the user interface 20 shown, if the text information 202A, the rejection control 202B, and the permission control 202C in the pop-up window 102 exceed the display area, the electronic device 100 can move the text information 202A to the right and the rejection control 202B and the permission control 202C to the left, so that the text information 202A, the rejection control 202B, and the permission control 202C can be fully displayed in the display area. In this way, the user can view the complete text information in the pop-up window 202, and can also choose whether to grant the application the permission to obtain the location information of the electronic device 100 through the rejection control 202B and the permission control 202C.

[0115] It is understandable that Figure 2A-2E For content not mentioned in the above, please refer to the Figure 1A-1E The contents in the above will not be repeated here. In addition, it should be noted that Figure 1B and Figure 2B The pop-up window shown is a pop-up window with a size that is too large and the displayed content exceeds the display area. Even if the pop-up window is only too large in size or the displayed content exceeds the display area, the method for adjusting the pop-up window proposed in this solution is applicable.

[0116] It can be seen that for a pop-up window in the user interface, if the pop-up window is too large and / or the content of the pop-up window exceeds the display area, etc., the electronic device 100 can adjust the pop-up window so that the user can view the complete content of the pop-up window, or facilitate the user to manipulate the pop-up window.

[0117] In addition, it should be noted that the above Figure 1A-1E , Figure 2A-2ETaking the electronic device 100 as a folding screen device as an example, the user interface involved in the electronic device 100 adjusting the incompatible pop-up window is shown. It should be understood that in addition to the folding screen device may display incompatible pop-up windows in the unfolded state scenario, incompatible pop-up windows may also appear in other scenarios. For example, when the tablet displays the application through a parallel field of view, or when the application on the mobile phone is projected to other devices (such as car-mounted devices, smart screens, etc.), the displayed pop-up windows may be incompatible. In other words, the embodiment of the present application does not limit the type of device. As long as the electronic device 100 has an incompatible pop-up window that needs to be displayed, the pop-up window can be adjusted by the pop-up window adjustment method provided in the embodiment of the present application.

[0118] The following describes the principle of adjusting a pop-up window by the pop-up window adjustment method provided in an embodiment of the present application.

[0119] In general, the pop-up window adjustment method provided in the embodiment of the present application mainly includes the following two steps: 1) finding the pop-up window that needs to be adjusted in the user interface; 2) adjusting the pop-up window.

[0120] Among them, in step 1, since the pop-up window is a window that is different from the application page and is additionally displayed on the application page, the electronic device 100 can monitor the layout changes of the interface elements contained in the user interface in real time, and search for the pop-up window that needs to be adjusted in the user interface according to the layout changes. Exemplarily, if it is detected that the layout changes of the interface elements contained in the user interface are large, it means that there may be a pop-up window newly displayed on the application page, and the search for the pop-up window can be triggered.

[0121] Figure 3 A schematic diagram of a process of the electronic device 100 monitoring the layout of the user interface is shown.

[0122] S101. The electronic device 100 records the number of interface elements in the user interface whose layout has changed.

[0123] The electronic device 100 may record in real time the number of interface elements in the user interface that have undergone layout changes during the process of displaying the user interface. In the embodiment of the present application, the user interface displayed by the electronic device 100 may also be referred to as a first user interface.

[0124] Interface elements (View or ViewGroup) are elements that make up the user interface. A user interface is composed of multiple interface elements, which can include: TextView, ImageView, ButtonView, etc. For example, see Figure 2A The text information 202A may refer to a TextView, and the reject control 202B and the allow control 202C may refer to a ButtonView.

[0125] The layout change of the interface elements includes the change of one or more of the display effects such as the position, size, color, etc. of the interface elements.

[0126] In order to better understand the relationship between interface elements and user interface, the following takes the Android system as an example to introduce the relationship between user interface and interface elements.

[0127] When drawing a user interface, the electronic device 100 will first obtain the layout file corresponding to the user interface, which records the interface elements contained in the user interface and the association relationship between these interface elements in the user interface. This association relationship is reflected by the view tree ViewTree corresponding to the user interface. ViewTree is a tree structure organized by interface elements in the Android user interface framework. In the ViewTree, an interface element (ViewGroup) can include child nodes (View or ViewGroup), and an interface element (View or ViewGroup) can also include a parent node (ViewGroup).

[0128] Figure 4 A schematic diagram of the structure of a ViewTree is shown as an example.

[0129] like Figure 4 As shown, the ViewTree is composed of interface elements such as DecorView, ViewGroup, and View, among which DecorView is the root node of ViewTree, has the highest level in ViewTree, and covers the entire display area of ​​the display screen. ViewGroup is the parent node of an interface element, and View is the child node of an interface element.

[0130] from Figure 4 It can be seen that the root node DecorView includes two child nodes (ViewGroup1, View2), ViewGroup1 includes two child nodes (View3, ViewGroup4), ViewGroup4 includes one child node (View5), and ViewGroup1 and View2 are sibling elements, and View3 and ViewGroup4 are sibling elements.

[0131] Before the electronic device 100 displays a user interface, the electronic device 100 draws each interface element layer by layer starting from the root node through the ViewTree corresponding to the user interface.

[0132] See also Figure 5, you can complete the drawing of the interface element by executing the three processes of measure, layout, and draw for the interface element in sequence. Among them, measure is used to measure the width and height of the View, layout is used to place the position of the View in the ViewGroup, and draw is used to draw the image on the View. In addition, the drawing of interface elements is a recursive process. The parent element draws itself and its child elements, and then the child element draws itself and its child elements.

[0133] Among them, in the embodiment of the present application, the electronic device 100 can record the interface elements whose layout changes in the user interface in the Layout process. In a specific implementation, the electronic device 100 can perform plugging in the Layout process, monitor the layout changes of the interface elements, and save the information of the interface elements whose layout changes in each frame into the set A. In this way, before drawing the next frame of the image, the number of interface elements saved in the set A is the number of interface elements whose layout changes in the current user interface in the previous frame.

[0134] S102. The electronic device 100 determines whether the number is greater than a threshold value 1.

[0135] Exemplarily, the threshold 1 may be 50 or 80, etc., and the embodiment of the present application does not limit the threshold 1.

[0136] If the number is greater than the threshold value 1, it means that the layout of the user interface drawn by the electronic device 100 has changed significantly, and there may be changes such as page switching, new pop-up windows, etc. In this way, the electronic device 100 can perform a search for pop-up windows in the user interface, that is, execute step S103 to determine whether there are pop-up windows in the user interface that need to be adjusted. Otherwise, the electronic device 100 can record and monitor the layout changes of the user interface.

[0137] In the embodiment of the present application, the threshold 1 may also be referred to as a second threshold.

[0138] S103. The electronic device 100 searches for a pop-up window in the user interface, and adjusts the pop-up window after finding the pop-up window that needs to be adjusted.

[0139] When the layout of the user interface changes significantly, the electronic device 100 can perform a search for pop-up windows in the user interface, that is, find the pop-up windows that need to be adjusted from the user interface, which may include: traversing each interface element in the user interface, and finding pop-up windows from the interface elements contained in the user interface according to the layout of the interface elements such as the position and size.

[0140] For a detailed description of step S103, please refer to the following Figure 6-Figure 14 The content of this article will not be expanded here.

[0141] S104. The electronic device 100 determines whether the search duration exceeds a preset duration.

[0142] The search duration refers to the time taken by the electronic device 100 to search for the pop-up window. For example, the preset duration is 30ms or 50ms, etc. The embodiment of the present application does not limit the preset duration.

[0143] In a specific implementation, after the electronic device 100 executes step S102, if the number is greater than threshold 1, the electronic device 100 can start the timer and, during the execution of step S103, determine whether the execution duration of step S103 exceeds the preset duration set by the timer.

[0144] If the execution time of step S103 exceeds the preset time, that is, the electronic device 100 searches for the pop-up window in the user interface for a longer time, the electronic device 100 executes step S105 to end the search for the pop-up window. Otherwise, the electronic device 100 may execute step S103 to continue searching for the pop-up window in the user interface.

[0145] In this way, it is possible to avoid the electronic device 100 spending too much time searching for pop-up windows, and control the power consumption of the electronic device 100.

[0146] It is understandable that step S104 is an optional step, and step S105 may be performed after the electronic device 100 performs step S103. That is, the electronic device 100 may end the search for the pop-up window after completing the search for the pop-up window. The completion of the search for the pop-up window here includes the two situations that the electronic device 100 finds the pop-up window in the user interface, and the electronic device 100 determines that the user interface does not contain the pop-up window that needs to be adjusted.

[0147] S105. The electronic device 100 ends the search for the pop-up window in the user interface.

[0148] Considering that the drawing of interface elements includes three processes: measure, layout, and draw, the execution timing of steps S101 to S105 may be as follows:

[0149] 1) In the embodiment of the present application, steps S101-S105 may be performed after the Layout process and before the Draw process.

[0150] In other words, the electronic device 100 can start monitoring the layout of the user interface before drawing the user interface each time, and when the layout changes significantly, perform a search for pop-up windows and adjust the pop-up windows that need to be adjusted. Since the draw process is used to draw images on interface elements to display interface elements, steps S101-S105 are performed after the Layout process and before the draw process, so that the electronic device 100 can find the pop-up window before displaying the pop-up window in the user interface, and determine in advance whether the pop-up window to be displayed is the pop-up window that needs to be adjusted, and adjust the pop-up window. In this way, when the electronic device 100 displays a pop-up window, the adjusted pop-up window can be directly displayed without displaying the pop-up window before adjustment. In this way, the user does not need to view pop-up windows of inappropriate size.

[0151] by Figure 1B As an example, the pop-up window 102 shown in FIG. 1 is used to display the electronic device 100. Figure 1B When the pop-up window 102 is displayed, the pop-up window 102 can be adjusted before the pop-up window 102 is displayed, so that the electronic device 100 can directly display the pop-up window 102. Figure 1C or Figure 1D or Figure 1E The pop-up window 102 shown will not be displayed Figure 1B An inappropriate pop-up window 102 is shown.

[0152] by Figure 2B As an example, the pop-up window 202 shown in FIG. 1 is used to display the electronic device 100. Figure 2B When the pop-up window 202 is displayed, the pop-up window 202 can be adjusted before the pop-up window 202 is displayed, so that the electronic device 100 can directly display the pop-up window 202. Figure 2C or Figure 2D or Figure 2E The pop-up window 102 shown will not be displayed Figure 2B An inappropriate pop-up window 202 is shown.

[0153] 2) In the embodiment of the present application, steps S101-S105 may be executed after the draw process.

[0154] In other words, the electronic device 100 can start monitoring the layout of the user interface after drawing the user interface each time, and when the layout changes significantly, perform a search for pop-up windows, and adjust the pop-up windows that need to be adjusted. In this case, the electronic device 100 can adjust the pop-up window after displaying the pop-up window that needs to be adjusted in the user interface. In this way, the electronic device 100 can display the pop-up window before and after the adjustment, and the user can view the dynamic adjustment process of the pop-up window.

[0155] by Figure 1BAs an example, the pop-up window 102 shown in FIG. Figure 1B After the pop-up window 102 is shown, the pop-up window 102 is adjusted, and then the pop-up window 102 is displayed again. Figure 1C or Figure 1D or Figure 1E Pop-up window 102 is shown.

[0156] by Figure 2B As an example, the pop-up window 202 shown in FIG. Figure 2B After the pop-up window 202 is shown, the pop-up window 202 is adjusted, and then the pop-up window 202 is displayed again. Figure 2C or Figure 2D or Figure 2E Pop-up window 202 is shown.

[0157] In a specific implementation, the electronic device 100 may perform function insertion between the Layout process and the draw process, or after the draw process, and the inserted function is used to implement steps S101-S105.

[0158] In general, monitoring the layout changes in the user interface and performing a search for pop-up windows when the layout changes significantly can avoid the electronic device 100 being in a state of searching for pop-up windows all the time. The characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application are fully considered. When it is determined that a pop-up window may appear in the user interface, a search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0159] Combine the following Figure 6-Figure 14 The details of the above step S103 are described in detail.

[0160] Figure 6 An overall flow chart for searching and adjusting pop-up windows provided in an embodiment of the present application.

[0161] like Figure 6 As shown, the search and adjustment pop-up window mentioned in the above step S103 mainly includes the following contents:

[0162] S201. The electronic device 100 searches the user interface for an interface element belonging to a pop-up window that needs to be adjusted.

[0163] In a specific implementation, the electronic device 100 can obtain all interface elements contained in the user interface through the layout file of the user interface. The layout file records the interface elements contained in the interface element and the association relationship between these interface elements in the user interface. The association relationship refers to the parent-child relationship, sibling relationship, etc. between nodes in the ViewTree. For a specific description of ViewTree, please refer to the aforementioned Figure 4 The relevant content will not be repeated here.

[0164] If a pop-up window is displayed in the user interface, the interface elements included in the user interface include one or more interface elements corresponding to the pop-up window.

[0165] For example, Figure 1A and Figure 1B Take the pop-up window 102 as an example. Figure 7 A schematic diagram showing the structure of a pop-up window in ViewTree.

[0166] like Figure 7 As shown, the pop-up window 102 may be composed of three interface elements: ViewGroup6, View7, and View8, wherein ViewGroup6 may refer to the background of the pop-up window 102, View7 may refer to the pop-up window main body 102A in the pop-up window 102, and View8 may refer to the close control 102B. ViewGroup6 is the parent element of View7 and View8.

[0167] It should be noted that the background of the pop-up window 102 can be an invisible transparent background, or a visible semi-transparent background, such as a gray semi-transparent background. Figure 7 The background of the pop-up window 102 is shown in the form of a dotted frame, which does not mean that the background of the pop-up window 102 is within the range where the dotted frame is located. For example, the range where the background of the pop-up window 102 is located may be the range where the display screen is located.

[0168] Among them, the pop-up window that needs to be adjusted is an inappropriate pop-up window, which may be a pop-up window that is too large and / or a part of the content exceeds the display area. Therefore, in the process of searching for pop-ups, the interface element in the user interface that belongs to the pop-up window that needs to be adjusted can be determined based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the user interface.

[0169] For a detailed description of the electronic device search pop-up window, please refer to the subsequent content, which will not be expanded here.

[0170] S202. The electronic device 100 adjusts the found pop-up window.

[0171] In the embodiment of the present application, the found pop-up window may also be referred to as the first pop-up window.

[0172] Among them, if the size of the pop-up window is too large, the electronic device 100 displays a pop-up window with a size greater than the threshold in the user interface before adjustment, and the adjustment may refer to reducing the pop-up window. If a part of the content of the pop-up window exceeds the display area, the electronic device 100 displays a part of the content of the pop-up window in the user interface before adjustment, and another part of the content, such as the first interface element, exceeds the display area of ​​the display screen. The adjustment may refer to reducing the pop-up window, or adding a new interface element in the display area, and the newly added interface element and the interface element in the pop-up window that exceeds the display area are used to trigger the same function, or the interface element that exceeds the display area is moved to the display area.

[0173] Exemplarily, the electronic device 100 may reduce the pop-up window in any of the following ways:

[0174] 1) Reduce the pop-up window according to the specified reduction ratio

[0175] For example, the electronic device 100 may reduce the pop-up window to 70% of the original size. The embodiment of the present application does not limit the reduction ratio.

[0176] 2) Reduce the size of the pop-up window until all of its contents are within the display area.

[0177] In this case, the reduction ratio of the pop-up window is not fixed, and the electronic device 100 can adaptively adjust the reduction ratio of the pop-up window according to the size of the pop-up window.

[0178] It is understandable that the embodiments of the present application do not limit the method of reducing the pop-up window.

[0179] It can be seen that reducing the size of the pop-up window can prevent the pop-up window from blocking too much content on the application page. Moreover, if the pop-up window displays content that exceeds the display area, reducing the size of the pop-up window can also make the entire content of the pop-up window as much as possible within the display area, making it easier for users to view the entire content of the pop-up window.

[0180] If the content beyond the display area in the pop-up window includes the first interface element, the newly added interface element in the display area, such as the second interface element, can be used to trigger the same function as the first interface element. For example, taking the first interface element as a close control, the newly added second interface element and the first interface element can both be used to trigger the electronic device 100 to close the pop-up window. In a specific implementation, the electronic device 100 can map the interaction event of the first interface element to the second interface element.

[0181] It can be seen that mapping the interaction events of the second interface element to the first interface element can ensure that the pop-up window located in the display area can contain complete interaction events as much as possible. In this way, even if the current pop-up window is not displayed completely, the pop-up window still contains complete functions, which is convenient for users to operate the pop-up window.

[0182] If the content beyond the display area in the pop-up window includes a first interface element, moving the interface element beyond the display area into the display area may refer to moving the first interface element into the display area, and the first interface element may refer to text, a picture, a control, etc. Among them, the electronic device 100 may move the first interface element to any position in the display area, and preferably, the first interface element may be moved to a blank area of ​​the pop-up window, which not only prevents the first interface element from blocking other content, but also allows the user to see that the first interface element belongs to the content in the pop-up window.

[0183] It can be seen that moving the interface elements that exceed the display area into the display area can ensure that the user can view the complete content in the pop-up window as much as possible, while also making it easier for the user to manipulate the pop-up window.

[0184] by Figure 1B As an example, the pop-up window 102 shown in FIG. 1 may be a pop-up window 102 of FIG. Figure 1C The pop-up window 102 shown in the figure, the first interface element can refer to Figure 1B The close control 102B shown (see Figure 1A The close control 102B shown in the figure), the second interface element may refer to Figure 1D The closing control 102C shown in the figure, the first interface element after the movement position may refer to Figure 1E Close control 102B is shown.

[0185] by Figure 2B As an example, the pop-up window 202 shown in the figure can be a reduced pop-up window. Figure 1C As shown in the pop-up window 202, the first interface element may refer to Figure 2B The permission control 202C shown (see Figure 2A The permission control 202C shown in FIG. 202C), the second interface element may refer to Figure 2D The permission control 202D shown, the first interface element before the movement can be seen Figure 2B The text information 202A, the reject control 202B, and the allow control 202C shown in FIG. Figure 2E Shown are text message 202A, a deny control 202B, and an allow control 202C.

[0186] In the embodiment of the present application, the search pop-up window mentioned in step S201 may include the following two methods:

[0187] 1) Find pop-ups by searching for whitelisted elements in the user interface

[0188] The whitelist is a configuration library created and maintained by developers, and includes information about interface elements contained in one or more pop-up windows, which may refer to element identifiers (IDs) of interface elements. Developers can generate the whitelist by collecting information about interface elements contained in pop-up windows that need to be adjusted and displayed in different applications.

[0189] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains an interface element on the whitelist.

[0190] The interface element in the whitelist may refer to the element with the highest level in the ViewTree among all the interface elements included in the pop-up window. Figure 7 Taking pop-up window 102 as an example, the highest-level element may refer to ViewGroup 6. Then, after electronic device 100 finds the interface element in the pop-up window from the user interface through the whitelist, all sub-elements of the found interface element may be determined as the interface element included in the pop-up window.

[0191] 2) Find the pop-up window by specifying the interface elements contained in the pop-up window

[0192] Since pop-up windows are different from application pages, they usually have some fixed features to distinguish them from application pages. For example, pop-up windows usually contain designated interface elements such as close controls, confirmation controls, cancel controls, reject controls, and allow controls, and the positions of these designated interface elements are relatively fixed. For example, the close control is usually located in the lower middle or upper right corner of the display screen, so that users can control the pop-up window through the designated interface elements.

[0193] It can also be understood that the specified interface element is a necessary interface element in the pop-up window. If the specified interface element exceeds the display area or exceeds the area that is convenient for the user to manipulate, it may affect the user's manipulation of the pop-up window, or affect the user's reading of the prompt information displayed in the pop-up window. Then, when searching for a pop-up window by specifying an interface element, if the specified interface element exceeds the display area or exceeds the area that is convenient for the user to manipulate, if it is determined that the specified interface element is an element in the pop-up window, then it can be determined that the pop-up window is a pop-up window that needs to be adjusted.

[0194] Therefore, when searching for a pop-up window on the electronic device 100, the pop-up window can be searched for by first searching for a designated interface element in the user interface that may belong to the pop-up window, and then searching for the pop-up window through the designated interface element.

[0195] For detailed information about method 2, please refer to the following Figure 8 The relevant content will not be expanded here.

[0196] In some implementations, the electronic device 100 may combine the above two methods to search for pop-up windows. For example, the electronic device 100 may first search whether the user interface contains an interface element in the whitelist through method 1, and if not, search whether the user interface contains a specified interface element through method 2, and then search for a pop-up window through the specified interface element found.

[0197] In addition, the information of the interface elements in each pop-up window can be continuously collected, and the whitelist can be updated regularly so that the electronic device 100 can directly use the whitelist to more accurately identify the pop-up window. Among them, the information of the collected interface elements can come from the interface elements in the pop-up window found by each electronic device using the above method 2, or from the pop-up window extracted by the developer when testing each application. For example, after each electronic device finds the pop-up window using the above method 2, the interface elements contained in the pop-up window can be sent to the server in a centralized manner, and the server collects the interface elements in the pop-up window identified by each device, and then updates these interface elements to the whitelist, and re-sends the updated whitelist to each electronic device, so that each electronic device can directly use method 1 to identify the pop-up window, and minimize the number of times the electronic device identifies the pop-up window using method 2, thereby reducing the amount of computing power of the electronic device.

[0198] Figure 8 A flow chart of an overall method for the electronic device 100 to search for a pop-up window through a specified interface element contained in the pop-up window is shown.

[0199] like Figure 8 As shown, searching for a pop-up window through a specified interface element contained in the pop-up window mainly includes:

[0200] S301. The electronic device 100 searches for a designated interface element in a pop-up window from interface elements included in the user interface.

[0201] Exemplarily, the designated interface element may refer to a landmark element in a pop-up window such as a close control, a confirmation control, a cancel control, a reject control, or an allow control.

[0202] The electronic device 100 can find the specified interface element from the interface elements included in the user interface according to multiple factors included in the specified interface element, such as ID, attribute, width, height, position, interaction event, etc.

[0203] Taking the designation of an interface element as a close control as an example, the electronic device 100 can search for a close control from all interface elements included in the user interface according to any one or more of the following requirements: the element ID contains the keyword "close", the position of the interface element is far from the center of the display screen, the interface element contains an interactive event, etc. Among all the interface elements, the interface element that meets the above requirements is the close control.

[0204] S302. The electronic device 100 searches for a pop-up window that needs to be adjusted and is contained in the user interface according to the designated interface element.

[0205] After the electronic device 100 finds the designated interface element that may belong to the pop-up window, the electronic device 100 can continue to search for interface elements that are associated with the designated interface element through the designated interface element, and then determine whether these elements are interface elements included in the pop-up window based on the layout of these associated interface elements, thereby realizing the search for the pop-up window.

[0206] For a detailed description of step S302, please refer to the following Fig.12 The content of this article will not be expanded here.

[0207] Among them, in step S301, since the user interface includes a large number of interface elements, in order to speed up the speed at which the electronic device 100 searches for the specified interface elements, when searching for the specified interface elements, the electronic device 100 can first filter out the interface elements that preliminarily meet the requirements of the specified interface elements from the interface elements of the user interface, and then search for the specified interface element from these interface elements that meet the requirements.

[0208] That is, in some implementations, the above step S301 may include two stages:

[0209] Phase 1: Filter out interface elements that meet preset requirements from the interface elements of the user interface

[0210] The preset requirement may refer to a requirement that all the specified interface elements contained in any pop-up window must meet.

[0211] For example, taking the designation of the interface element as any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control as an example, the requirement may mean that the interface element includes an interactive event. This is because these interface elements may have different positions and forms in the pop-up window, but they must all include an interactive event, which is used to enable the electronic device 100 to trigger a certain function according to the user operation acting on the close control, such as the close control is used to trigger the close pop-up window.

[0212] Therefore, the electronic device 100 may first filter out a portion of interface elements according to certain requirements, and then search for a specified interface element from the filtered interface elements. This can reduce the amount of computation of the electronic device 100 and speed up the search of the electronic device 100.

[0213] In a specific embodiment, Fig. 9 A detailed flow chart of the electronic device 100 screening out interface elements that meet the requirements from the interface elements of the user interface is shown.

[0214] like Fig. 9 As shown, stage one may include the following steps:

[0215] S401. The electronic device 100 traverses the interface elements in the user interface from the bottom layer of the ViewTree of the user interface upwards.

[0216] The ViewTree of the user interface records all the interface elements contained in the user interface. The electronic device 100 can traverse the interface elements in the user interface from the bottom layer of the ViewTree of the user interface, and then determine the interface elements that belong to the pop-up window in the first user interface based on any one or more of the above-mentioned information of the traversed interface elements.

[0217] This is because if the user interface includes a pop-up window, in the ViewTree, the interface elements closer to the lower layer are drawn later and are more likely to be pop-ups. Therefore, traversing upward from the bottom layer of the ViewTree can minimize the number of interface elements traversed by the electronic device 100, and when there is a pop-up window in the user interface, the interface element that meets the requirements can be found as soon as possible.

[0218] It is understandable that the electronic device 100 may also traverse downward from the topmost root node of the ViewTree, which is not limited in the embodiment of the present application.

[0219] S402. The electronic device 100 determines whether the currently traversed interface element is in the whitelist.

[0220] The whitelist may be a configuration library preset by the developer in the electronic device 100. The whitelist may be used to record the highest-level elements in the ViewTree among all the interface elements contained in the pop-up window. For example, the electronic device 100 may determine whether the currently traversed interface element is an interface element in the pop-up window by determining whether the element ID of the currently traversed interface element is in the whitelist.

[0221] If the currently traversed interface element is in the whitelist, the electronic device 100 executes step S403 , otherwise, executes step S404 .

[0222] S403. The electronic device 100 ends the traversal and determines all sub-elements of the currently traversed interface element as interface elements included in the pop-up window.

[0223] If the interface element currently traversed by the electronic device 100 (for example, the third interface element) is in the whitelist, it means that the user interface contains a pop-up window, and the currently traversed interface element is the highest-level element in the ViewTree contained in the pop-up window.

[0224] For example, Figure 7 Taking the pop-up window shown as an example, the currently traversed interface element may refer to ViewGroup6. View7 and View8, as child elements of ViewGroup6, may be determined as the interface elements contained in the pop-up window.

[0225] It is understandable that steps S402 and S403 are optional steps. If the electronic device 100 does not have a whitelist preset, the electronic device 100 can directly search for interface elements in the user interface that meet the preset requirements, that is, execute step S404.

[0226] S404. The electronic device 100 determines whether the currently traversed interface element meets the preset requirements.

[0227] If the interface element currently traversed by the electronic device 100 is not in the whitelist, the electronic device 100 may further determine whether the interface element meets the preset requirements.

[0228] Among them, the preset requirement may mean that the interface element contains an interactive event, that is, the electronic device 100 can detect the user operation acting on the interface element and trigger a corresponding response.

[0229] If the currently traversed interface element meets the preset requirements, the electronic device 100 executes step S405 to save the interface element into set B. Otherwise, the electronic device 100 executes step S406 to further determine whether the screen ratio of the currently traversed interface element is greater than a threshold.

[0230] S405. The electronic device 100 saves the interface element into set B.

[0231] The electronic device 100 saves the interface element into set B, so that set B can collect interface elements in the user interface that meet preset requirements.

[0232] S406. The electronic device 100 determines whether the screen ratio of the currently traversed interface element is greater than threshold 2.

[0233] Exemplarily, the threshold 2 may refer to 60%-80%, and the embodiment of the present application does not limit the threshold 2.

[0234] If the screen ratio of the currently traversed interface element is greater than the threshold 2, the electronic device 100 executes step S407 to end the traversal; otherwise, the electronic device 100 executes step S401 to continue traversing the next interface element.

[0235] That is to say, when the electronic device 100 is searching for an interface element that meets preset requirements in the user interface, if an interface element with a larger size is found, the traversal is terminated.

[0236] In the embodiment of the present application, the threshold 2 may also be referred to as a third threshold.

[0237] For example, Figure 7 Taking the pop-up window shown as an example, the larger interface element may refer to View7.

[0238] In this way, if the electronic device 100 traverses from the bottom layer of the ViewTree upwards, when the electronic device 100 finds an interface element of a larger size, it means that the electronic device 100 may have traversed most or all of the interface elements in the pop-up window. The electronic device 100 can end the traversal to avoid saving the interface elements that do not belong to the pop-up window into the set B, saving search time.

[0239] S407. The electronic device 100 ends the traversal.

[0240] It can be seen from steps S401 to S407 that the electronic device 100 can find one or more interface elements that meet preset requirements by traversing the interface elements in the user interface.

[0241] Stage 2: Find the specified interface elements in the pop-up window from the interface elements that meet the preset requirements

[0242] Since multiple interface elements meeting the preset requirements may be found in stage one, the electronic device 100 may further combine other possible features of the specified interface element to find the element that best meets the features of the specified interface element from among the multiple interface elements meeting the preset requirements.

[0243] For example, Fig.10 A schematic diagram of a process for searching for a specified interface element provided in an embodiment of the present application.

[0244] like Fig.10 As shown, stage two may include the following steps:

[0245] S501. The electronic device 100 determines the scores of the interface elements that meet the preset requirements.

[0246] The electronic device 100 may use a scoring rule to determine a score of an interface element that meets preset requirements, and the score indicates the possibility that the interface element is a designated interface element.

[0247] In the scoring system, multiple rules can be established, which describe the features that the specified interface element may have and the features that it does not have. The score of the interface element can be determined by comparing whether the interface element meets these features and evaluating the differences between these interface elements and the specified layout elements in terms of element ID, attributes, width, height, position, etc. If the interface element contains the features that the specified interface element may have, then the interface element will be given extra points, and if the interface element contains the features that the specified interface element may not have, then the interface element will be deducted points.

[0248] In this way, the scoring rule can be used to determine the possibility that each interface element that meets the preset requirements is a designated interface element, thereby realizing the search for the designated interface element from these interface elements that meet the preset requirements. In addition, the use of the scoring rule to search for the designated interface element can combine multiple factors and evaluate the similarity between the found interface element and the designated interface element from multiple aspects, thereby improving the accuracy of searching for the designated interface element.

[0249] S502. The electronic device 100 determines whether the interface elements that meet the preset requirements include interface elements with scores greater than a threshold value of 3.

[0250] Exemplarily, the threshold value 3 may be 0. The embodiment of the present application does not limit the threshold value 3.

[0251] If the interface elements meeting the preset requirements include interface elements with scores greater than the threshold value 3, the electronic device 100 executes step S503; otherwise, executes step S504.

[0252] It is understandable that step S502 is an optional step, and after determining the scores of each interface element, the electronic device 100 can directly determine the interface element with the highest score as the designated interface element. Step S502 can improve the evaluation criteria for the designated interface element and improve the accuracy of finding the designated interface element.

[0253] S503. The electronic device 100 determines the interface element with the highest score as the designated interface element.

[0254] The interface element with the highest score indicates that the interface element is most likely to be the designated interface element. In other words, the electronic device 100 may determine the interface element with the highest score and a score greater than a threshold value 3 as the designated interface element.

[0255] S504. The electronic device 100 determines that the user interface does not include a pop-up window that needs to be adjusted.

[0256] If the interface elements meeting the preset requirements do not include interface elements with scores greater than the threshold 3, it means that the interface elements meeting the preset requirements do not include the specified interface element, that is, the user interface does not include the pop-up window that needs to be adjusted.

[0257] In this way, the electronic device 100 can end the search for the pop-up window and avoid mistaking elements similar to the specified interface elements as the specified interface elements as much as possible, thereby saving the electronic device 100 from subsequent invalid operations of searching for pop-up windows based on the specified interface elements.

[0258] It is understandable that the electronic device 100 may also skip the process of screening layout elements that meet preset requirements in stage 1, and directly execute stage 2 to calculate the scores of each interface element in the user interface, and find the specified interface element according to the score.

[0259] In order to better understand steps S501-S504, the following takes the designation of an interface element as a close control as an example to describe a specific example of the electronic device 100 using a scoring rule to find a close control from multiple interface elements.

[0260] Exemplarily, Table 1 shows a scoring rule table for finding a closed control provided in an embodiment of the present application.

[0261] Table 1

[0262]

[0263] Among them, in the above Table 1:

[0264] (1) The element attribute value in rule 2 can be obtained through the getContentDescription() method, which can obtain the contentDescription attribute value of the interface element. Among them, this attribute value is mainly used in barrier-free services to add explanations of controls for people with visual impairments. For example, when a user clicks on a control through barrier-free services, the electronic device 100 can extract the contentDescription attribute value of the control through the getContentDescription() method and play the voice, so that the user can determine the control clicked by the user by listening to the voice.

[0265] (2) If the interface element is a TextView, the interface element itself has a Text attribute, which can carry some explanatory text labels.

[0266] It is understandable that if the interface element is a TextView, Rule 3 can be used to determine whether the interface element is related to keywords such as "close", "abandon", "skip", "reject", etc., without having to repeat the judgment through Rule 2. If the interface element is not a TextView, Rule 2 can be used to determine whether the interface element is related to keywords such as "close", "abandon", "skip", "reject", etc. In this way, the amount of calculation of the electronic device 100 can be reduced. Therefore, when using the scoring rule table shown in Table 1 above to determine the score of the interface element, in addition to using Rule 1 and Rules 4-6 to determine the score of the interface element, you can choose to calculate the score of the interface element from either Rule 2 or Rule 3.

[0267] (3) The width or height of the interface element in rule 4 is relatively large, which may mean that the width or height of the interface element is greater than threshold 4. For example, threshold 4 may be 350px or 400px. The embodiment of the present application does not limit threshold 4.

[0268] (4) The aspect ratio or height-to-width ratio of the interface element in rule 5 is large, which may mean that the aspect ratio or height-to-width ratio of the interface element is greater than a threshold value 5. For example, the threshold value 5 may be 1.3. The embodiment of the present application does not limit the threshold value 5.

[0269] (5) In Rule 6, since the close control of the pop-up window is usually located in the lower middle or upper right corner of the display screen, the possibility that the interface element is a close control can be determined based on the position of the interface element in the display screen.

[0270] For example, the display screen can be divided into nine 3*3 areas. Fig.11 A schematic diagram showing a display screen divided into nine areas.

[0271] See also Fig.11 The display screen may include nine areas, area 1 to area 9. Area 3 is the upper right corner area of ​​the display screen, and area 8 is the lower middle area of ​​the display screen. Rule 6 may mean that the interface elements located in area 3 or area 8 can be scored, and the farther away from the center point of area 5, the more points the interface elements can be scored.

[0272] In addition, it should be noted that the points subtracted by Rule 4 or Rule 5 can change dynamically. For example, in Rule 4, if the width or height of the interface element is greater than the threshold 4' and less than the threshold 4", 1 point is subtracted; if the width or height of the interface element is greater than the threshold 4", 2 points are subtracted, and so on. In other words, the larger the width or height of the interface element, the more points are subtracted. For another example, in Rule 5, if the aspect ratio or height-to-width ratio of the interface element is greater than the threshold 5' and less than the threshold 5", 1 point is subtracted; if the width or height of the interface element is greater than the threshold 5", 2 points are subtracted, and so on. In other words, the larger the aspect ratio or height-to-width ratio of the interface element, the more points are subtracted.

[0273] It can be understood that Table 1 is only an illustrative introduction to the example application of the scoring system rules in searching for specified interface elements. In the embodiment of the present application, when searching for specified interface elements, the scoring system rules may also include more or fewer rules, and the rules set in the scoring system rules may also be different for different specified interface elements to be searched, and the embodiment of the present application does not limit this.

[0274] After finding the designated interface element, the electronic device 100 may search for the pop-up window that needs to be adjusted and is contained in the user interface according to the designated interface element.

[0275] Fig.12 A flowchart of a method for searching for a pop-up window based on a specified interface element provided in an embodiment of the present application.

[0276] like Fig.12 As shown, searching for a pop-up window according to a specified layout space may include the following steps:

[0277] S601. The electronic device 100 searches for a parent element of a specified interface element in the user interface.

[0278] Exemplarily, the electronic device 100 can search for the parent element of the specified interface element from the ViewTree corresponding to the user interface. The parent element of the specified interface element may include multiple parent elements. The parent element may refer to a direct parent element or an indirect parent element. The electronic device 100 can search for the parent element of the specified interface element by traversing the ViewTree.

[0279] by Figure 4 For example, assuming that View5 is a designated interface element, the parent elements of View5 may include ViewGroup4, ViewGroup1, and DecorView, wherein ViewGroup4 is the direct parent element of View5, and ViewGroup1 and DecorView are indirect parent elements of View5.

[0280] S602. The electronic device 100 determines whether the parent element of the specified interface element includes an interface element that does not meet the conditions.

[0281] Among them, it can be determined whether all parent elements of the specified interface element include interface elements that do not meet the conditions, and it can also be determined whether N parent elements of the specified interface element include interface elements that do not meet the conditions, wherein the N parent elements can refer to the N parent elements traversed upward from the specified interface element in the ViewTree.

[0282] The electronic device 100 may analyze whether the parent element of the designated interface element includes interface elements that do not meet the conditions by comparing the page layouts of the interface elements.

[0283] Among them, page layouts of interface elements that do not meet the conditions include: waterfall flow layout, scroll page layout, Tab page layout, etc.

[0284] These layouts all belong to the layout of the application page, not the layout of the pop-up window. If the parent element of the specified interface element includes an interface element that does not meet the conditions, it means that the specified interface element is not an interface element in the pop-up window, so the electronic device 100 can end the search for the pop-up window; if the parent element of the specified interface element does not include an interface element that does not meet the conditions, it means that the specified interface element may be an interface element in the pop-up window, so the electronic device 100 can further search for other interface elements that may be in the pop-up window to determine whether the specified interface element is an element in the pop-up window.

[0285] Therefore, if the parent element of the designated interface element includes an interface element that does not meet the condition, the electronic device 100 executes step S606; otherwise, the electronic device 100 executes step S603.

[0286] S603. The electronic device 100 searches for multiple interface elements that are associated with the specified interface element or the interface element whose screen occupancy ratio is greater than a threshold value 2 found when traversing the interface elements, and determines the multiple interface elements as an element group.

[0287] These multiple interface elements can be regarded as the currently found interface elements that may be included in the pop-up window. In the embodiment of the present application, the element group composed of these multiple interface elements can also be called the first element group.

[0288] The electronic device 100 may search for the multiple interface elements in any of the following ways:

[0289] 1) Find these multiple interface elements according to the specified interface element

[0290] The multiple interface elements may include: a designated interface element, a sibling element of the designated interface element, a child element of the designated interface element, and a child element of the sibling element.

[0291] It is understandable that if the specified interface element does not contain child elements, then the multiple interface elements may not contain child elements of the specified interface element. Similarly, if the sibling element does not contain child elements, then the multiple interface elements may not contain child elements of the sibling element.

[0292] by Figure 7 Taking the pop-up window shown as an example, if the specified interface element is View8, the multiple interface elements may include: View7, View8.

[0293] 2) Find these multiple interface elements based on the interface elements whose screen ratio is greater than the threshold 2 found when traversing the interface elements

[0294] Among them, the multiple interface elements may include all child elements of the parent element of the interface element whose screen ratio is greater than threshold 2.

[0295] by Figure 7 Taking the pop-up window shown as an example, if the interface element whose screen proportion is greater than threshold 2 is View7, then the multiple interface elements may include: View7, View8.

[0296] It is understandable that the electronic device 100 can also find these multiple interface elements in other ways, and the embodiment of the present application is not limited to this.

[0297] S604. The electronic device 100 determines whether the element group meets the layout conditions of the pop-up window that needs to be adjusted.

[0298] Among them, the electronic device 100 can judge whether the element group meets the layout conditions of the pop-up window that needs to be adjusted based on the gap between the position, type, interactive event, width, height and other factors of each interface element in the element group and the pop-up window included in the pop-up window that needs to be adjusted.

[0299] Specifically, the electronic device 100 may use a scoring rule similar to that mentioned in step S501 to determine the score of the element group, which indicates the possibility that the element group is a pop-up window that needs to be adjusted. In other words, if the score meets certain requirements, the element group meets the layout conditions of the pop-up window that needs to be adjusted.

[0300] Illustratively, Table 2 shows a scoring rule table for evaluating whether an element group meets the layout conditions of a pop-up window that needs to be adjusted, provided in an embodiment of the present application.

[0301] Table 2

[0302]

[0303] Among them, in the above Table 2:

[0304] (1) In rule 1, if there are interface elements in the element group that exceed the screen display area, it means that there are some contents in the element group that are invisible to the user.

[0305] Furthermore, the rule 1 can be further limited to the specified interface element in the element group exceeding the screen display area. In this case, even if the element group is a pop-up window and some of the content exceeds the screen display area, the electronic device 100 will not identify it as a pop-up window that needs to be adjusted. Taking the specified interface element as a close control in a pop-up window as an example, if there is text content in the pop-up window that exceeds the display area, but the close control does not exceed the display area, although the user cannot view the complete content in the pop-up window, the pop-up window can still be quickly closed by closing the control.

[0306] (2) In rule 2, the complex layout of the element group may mean that the number of element types contained in the element group exceeds a threshold of 6. For example, the threshold of 6 may be 10. This is because only limited content can be displayed in a pop-up window. Generally speaking, a pop-up window contains fewer element types. Therefore, if the layout of the element group is complex, the possibility that the element group is a pop-up window can be reduced.

[0307] In addition, the specified type of interface elements may include: radio buttons, check boxes, edit input boxes, etc. This is because pop-up windows generally do not contain radio buttons, check boxes, edit input boxes, etc. interface elements, so if the element group contains these types of interface elements, it can reduce the possibility that the element group is a pop-up window.

[0308] (3) In rule 3, there are too many interface elements with interactive events in the element group, which may mean that the number of interface elements with interactive events in the element group is greater than a threshold value 7. For example, the threshold value 7 may be 4. This is because a pop-up window generally contains only a few controls such as a close control, a cancel control, and a confirmation control for interacting with the user. If there are too many interface elements with interactive events in the element group, the possibility that the element group is a pop-up window can be reduced.

[0309] (4) Rule 4 corresponds to Rule 3, and the number of interface elements with interactive events is within a certain range, for example, the number does not exceed threshold 8. The threshold 8 may be equal to threshold 7 or less than threshold 7. In other words, if there are fewer interface elements with interactive events in the element group, the possibility that the element group is a pop-up window may be increased.

[0310] (5) In rule 5, if the element group contains a large number of small-sized interface elements, the element group may not have a large-sized problem, and the pop-up window is generally displayed in the middle of the display screen. If some of the interface elements in the element group are not centered, the possibility that the element group is a pop-up window can be reduced. Among them, small size can refer to the width and / or height of the interface element being less than threshold 9, such as 350px, 400px. The embodiment of the present application does not limit the threshold 9. In addition,

[0311] (6) Rule 6 corresponds to Rule 5. If the element group contains a large number of large-sized interface elements, the size of the element group may be large, which increases the possibility that the element group is a pop-up window that needs to be adjusted.

[0312] It can be understood that Table 2 is only an illustrative introduction to the example application of the scoring system rules in finding pop-up windows that need to be adjusted. In the embodiment of the present application, when judging whether an element group is a pop-up window that needs to be adjusted, the scoring system rules can also include more or fewer rules, and the scoring system rules can include other different rules, which is not limited in the embodiment of the present application.

[0313] Further, after the score of the element group is calculated by a scoring rule similar to that in Table 2, the score can be used to indicate the possibility that the element group is a pop-up window that needs to be adjusted. The higher the score, the higher the possibility, and the lower the score, the lower the possibility. If the score of the element group exceeds the threshold 9, for example, 0, it is determined that the element group is a pop-up window that needs to be adjusted, that is, step S605 is executed. Otherwise, the user interface does not contain a pop-up window that needs to be adjusted, that is, step S606 is executed.

[0314] S605. The electronic device 100 determines the element group as a pop-up window that needs to be adjusted.

[0315] The electronic device 100 determines the element group as a pop-up window that needs to be adjusted, indicating that the element group belongs to the elements in the pop-up window that needs to be adjusted. After the electronic device 100 determines the element group as a pop-up window that needs to be adjusted, the electronic device 100 can adjust the element group to adjust the pop-up window.

[0316] Among them, the adjustment of the pop-up window may include: 1) reducing the pop-up window; 2) adding a mapping interface element; 3) adjusting the position of the interface element. Specifically, reducing the pop-up window may refer to reducing each element in the element group and moving it to the center point of the element group; adding a mapping interface element may refer to adding a new interface element to the user interface, and adding the interaction event of the interface element in the element group that is outside the display area to the newly added interface element; adjusting the position of the interface element may refer to moving the interface element in the element group that is outside the display area to the display area content. For details about the adjustment of the pop-up window, please refer to the aforementioned content, which will not be repeated here.

[0317] In addition, for pop-up windows where some interface elements are located outside the main content of the pop-up window, before shrinking the pop-up window, the pop-up window size is enlarged to fit the enlarged display area. Therefore, during the enlargement of the pop-up window, the main content of the pop-up window may squeeze the display area of ​​the part of the interface element, resulting in the height or width of some interface elements being 0. In this case, before shrinking the pop-up window, you also need to adjust the height or width of this part of the interface element, so that the height or width of this part of the interface element is greater than 0, and then shrink the pop-up window.

[0318] For example, this part of the interface elements includes Figure 1A and Figure 1B As an example, see the close control 102B shown in FIG. Figure 1B As shown in the pop-up window 102, since the close control 102B is located below the pop-up window main part 102A, when the electronic device 100 draws Figure 1B In the pop-up window 102 shown, the pop-up window main body 102A will vertically squeeze the close control 102B, causing the height of the close control 102B to be 0, so that the pop-up window 102 displayed by the electronic device 100 does not include the close control 102B.

[0319] The following takes the electronic device 100 adjusting the height of the interface element as an example to describe the specific process of reducing the pop-up window of the electronic device 100.

[0320] in, Fig.13 A schematic diagram of the process of reducing the pop-up window provided in an embodiment of the present application.

[0321] like Fig.13 As shown, reducing the pop-up window may include the following steps:

[0322] S701. The electronic device 100 determines whether the height of the interface element outside the display area is 0.

[0323] In the embodiment of the present application, the interface element located outside the display area may refer to a first interface element.

[0324] If the height of the interface element is 0, the electronic device 100 executes step S702 , otherwise, the electronic device 100 executes step S703 .

[0325] S702. The electronic device 100 adjusts the height of the interface element.

[0326] Specifically, the electronic device 100 can increase the height of the interface element. Accordingly, the electronic device 100 needs to reduce the height of other interface elements in the pop-up window that are arranged vertically with the interface element, so that the interface element and other interface elements in the pop-up window can be displayed simultaneously in the display area of ​​the pop-up window.

[0327] S703. The electronic device 100 reduces the size of the pop-up window.

[0328] In a specific implementation, the electronic device 100 may traverse each element in the element group, and scale each interface element in the element group based on the center point of the parent element of the topmost element in the element group, thereby reducing the pop-up window.

[0329] For example, Figure 1A and 1B Take the pop-up window 102 as an example. Fig.14 A schematic diagram showing the process of the electronic device 100 adjusting the height of the interface element and reducing the pop-up window.

[0330] like Fig.14 As shown in (a), the main part 102A of the pop-up window occupies the display area of ​​the pop-up window, and the close control 102B exceeds the display area of ​​the pop-up window. At this time, the height of the close control 102B in the pop-up window display area is 0.

[0331] like Fig.14 As shown in (b), the electronic device 100 adjusts the relative heights of the pop-up window main body 102A and the close control 102B, so that the height of the close control 102B in the pop-up window display area is greater than 0.

[0332] like Fig.14 As shown in (c) of FIG. 1 , the electronic device 100 is further reduced in size. Fig.14 The pop-up window shown in (b) enables the electronic device 100 to display the complete pop-up window on the display screen.

[0333] It is understandable that part of the content of the pop-up window exceeds the display area, and the main content of the pop-up window may also squeeze other interface elements horizontally, causing the width of some interface elements to be 0. Therefore, the electronic device 100 can also determine whether the width of the interface elements that exceed the display area is 0. If the width of the interface element is 0, the electronic device 100 first adjusts the width of the interface element and then reduces the size of the pop-up window. The adjustment process is similar to steps S701-S703 and will not be repeated here.

[0334] In addition, Fig.14 In (b), if the electronic device 100 further adjusts the relative heights of the pop-up window main body 102A and the close control 102B so that the close control 102B can be completely located within the pop-up window display area, the electronic device 100 can display the complete pop-up window without scaling the pop-up window.

[0335] That is to say, step S703 is an optional step. The electronic device 100 can make the interface element that exceeds the display area be located in the display area by adjusting the relative height or relative width between the interface element that exceeds the display area and other interface elements in the pop-up window, thereby omitting the step of scaling the pop-up window by the electronic device 100 and reducing the computational complexity of the electronic device 100.

[0336] S606. The electronic device 100 ends the search for the pop-up window.

[0337] It is understandable that after the electronic device 100 finishes searching for the pop-up window, the electronic device 100 may continue to monitor the layout changes in the currently displayed user interface, and trigger the search for the pop-up window when the layout changes significantly.

[0338] In addition, it should be noted that in the embodiment of the present application, the pop-up window adjustment method can be a system function in the electronic device 100. In this way, the electronic device 100 does not need to pre-install third-party applications in advance. The pop-up window in any application can be searched and adjusted directly through the system unit, which has a wider range of application scenarios and also facilitates user operations.

[0339] In general, the pop-up window adjustment method provided in the embodiment of the present application can find the pop-up window that needs to be adjusted in the user interface, and adjust the pop-up window so that the adjusted pop-up window has a more appropriate size, or can make it easier for users to control the pop-up window, thereby ensuring the continuity of users' browsing of page content, making the page content more beautiful, and improving the user's experience.

[0340] Fig.15 A flowchart of a pop-up window adjustment method provided in an embodiment of the present application.

[0341] like Fig.15 As shown, the pop-up window adjustment method may include:

[0342] S801. The electronic device 100 displays a first pop-up window, and a first interface element of the first pop-up window exceeds a display area of ​​the display screen.

[0343] Exemplarily, the first pop-up window may refer to Figure 1B As shown in the pop-up window 102, the first interface element may refer to Figure 1B The close control 102B shown, or the first pop-up window may refer to Figure 2B As shown in the pop-up window 202, the first interface element may include Figure 2B The text information 202A and the permission control 202C are shown.

[0344] S802. The electronic device 100 adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window, or displaying a second interface element in the display area, where the second interface element and the first interface element are used to trigger the same function, or moving the first interface element into the display area.

[0345] For example, the first pop-up window after shrinking may refer to Figure 1C In the pop-up window 102 shown, the second interface element may refer to Figure 1D The close control 102C shown, or the first pop-up window after shrinking can refer to Figure 2C In the pop-up window 202 shown, the second interface element may refer to Figure 2D An allow control 202C is shown.

[0346] In some embodiments, when the electronic device 100 displays the first pop-up window, if the height of the first interface element is 0, the electronic device 100 may increase the height of the first interface element before reducing the first pop-up window.

[0347] Among them, increasing the height of the interface element before shrinking the pop-up window can ensure that the first interface element can be displayed in the user interface after the pop-up window is shrunk, ensuring that the electronic device 100 can adjust the content of the pop-up window located outside the display screen to within the range of the display screen.

[0348] For a detailed description of how the electronic device 100 increases the height of the first interface element, please refer to the related description of the aforementioned steps S701-S703.

[0349] In addition, the first pop-up window can be displayed in the first user interface. Before the electronic device 100 adjusts the first pop-up window, the method also includes: the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, that is, searches for the first pop-up window in the first user interface.

[0350] That is to say, the electronic device 100 can evaluate the information of each interface element in the user interface to find the first pop-up window from the user interface, and then successfully adjust the pop-up window.

[0351] For a detailed description of the electronic device 100 searching for the first pop-up window, please refer to the aforementioned step S201 and its related contents.

[0352] Exemplarily, the first user interface may refer to Figure 1B The user interface 10 shown may also be referred to as Figure 2B The user interface 20 is shown.

[0353] Further, in some embodiments, when the electronic device 100 displays the first pop-up window, the number of interface elements in the first user interface that undergo layout changes is greater than a second threshold, and the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window, which may specifically include: when the number of interface elements in the first user interface that undergo layout changes is greater than the second threshold, the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window. Exemplarily, the second threshold may refer to the threshold 1 mentioned in the aforementioned step S102.

[0354] In this way, the electronic device 100 can monitor the layout changes of the interface elements contained in the user interface in real time, and search for pop-up windows that need to be adjusted in the user interface based on the layout changes. This can avoid the electronic device 100 being in a state of searching for pop-up windows all the time. The characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application are fully considered. When it is determined that a pop-up window may appear in the user interface, a search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0355] Furthermore, in some implementations, the electronic device 100 may determine the interface elements in the first interface elements that belong to the first pop-up window within a preset time period.

[0356] In this way, it is possible to avoid the electronic device 100 spending too much time searching for pop-up windows, and control the power consumption of the electronic device 100.

[0357] In some embodiments, the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface. Specifically, it may include: the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward, and the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

[0358] That is to say, when traversing the interface elements, the electronic device 100 can traverse from the bottom layer of the ViewTree upwards. This is because if the user interface includes pop-up windows, in the ViewTree, the interface elements that are closer to the lower layer are drawn later and are more likely to be pop-up windows. Therefore, traversing from the bottom layer of the ViewTree upwards can minimize the number of interface elements traversed by the electronic device 100. When there are pop-up windows in the user interface, the interface elements that meet the requirements can be found as quickly as possible, thereby improving the efficiency of the electronic device 100 in finding pop-up windows.

[0359] Furthermore, in some embodiments, after the electronic device 100 traverses the interface elements in the first user interface from the bottom layer of the ViewTree of the first user interface upward, the electronic device 100 may stop traversing after traversing to an interface element whose size is greater than a third threshold value. The third threshold value may refer to the threshold value 2 mentioned in the aforementioned step S406.

[0360] That is to say, when the electronic device 100 is traversing the interface elements, if it finds an interface element of a larger size, it can stop traversing. This is because if the user interface contains a pop-up window, when the electronic device 100 traverses to an interface element of a larger size, it means that most or all of the interface elements in the pop-up window may have been traversed. The electronic device 100 can end the traversal to avoid introducing interface elements that do not belong to the pop-up window into the calculation process during the search process, thereby saving search time.

[0361] Specifically, in the process of the electronic device 100 determining the interface element belonging to the first pop-up window in the first user interface, there may be the following two implementation methods:

[0362] 1) Search for pop-up windows through the whitelist

[0363] In this manner, the electronic device 100 may be pre-installed with a whitelist, which includes the IDs of the interface elements contained in one or more pop-up windows.

[0364] Then, the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface based on one or more information of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface. Specifically, it may include: the electronic device 100 finds the third interface element in the first user interface that is in the whitelist; the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface through the third interface element.

[0365] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains an interface element in the whitelist.

[0366] Exemplarily, the third interface element may be an interface element corresponding to the background contained in the first pop-up window, so that the interface element contained in the first pop-up window may be a child element of the third interface element. In other words, the third interface element may be the element with the highest level in the ViewTree of the first user interface among all the interface elements contained in the first pop-up window.

[0367] For example, see Figure 7 , with the first pop-up window as Figure 1B As an example of the pop-up window 102 shown in FIG. 1 , the third interface element may refer to Figure 7ViewGroup6 shown.

[0368] 2) Find pop-up windows by specifying interface elements

[0369] In this manner, the electronic device 100 determines the interface element in the first user interface that belongs to the first pop-up window based on one or more information of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface. Specifically, it may include: the electronic device 100 searches for a specified interface element based on the multiple information of the interface elements contained in the first user interface; the specified interface element is the interface element contained in the pop-up window; thereafter, the electronic device 100 can determine the interface element in the first user interface that belongs to the first pop-up window through the specified interface element.

[0370] For example, Figure 1B Taking the pop-up window 102 as an example, the designated interface element may refer to a close control 102B.

[0371] That is, the electronic device 100 may first search for a landmark element in the pop-up window and then use the landmark element to search for the pop-up window, so that the electronic device 100 can quickly find the pop-up window.

[0372] Further, in some embodiments, the electronic device 100 searches for a specified interface element based on the multiple information of the interface elements contained in the first user interface, which may specifically include: the electronic device 100 determines the scores of multiple interface elements in the first user interface based on the search of the multiple information of the interface elements contained in the first user interface and the multiple information of the specified interface element, and the score of an interface element indicates the possibility that the interface element is the specified interface element; thereafter, the electronic device 100 may search for the specified interface element from among the multiple interface elements based on the scores of the multiple interface elements.

[0373] Among them, the electronic device 100 calculates the score of the interface element through multiple information of the interface element, and can combine multiple factors to evaluate the similarity between the found interface element and the specified interface element from multiple aspects, thereby improving the accuracy and comprehensiveness of searching for the specified interface element.

[0374] For a detailed description of how the electronic device 100 searches for a specified interface element based on the score of the interface element, please refer to the related contents of the aforementioned steps S501-S504.

[0375] Exemplarily, the designated interface element may be any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before determining the scores of the multiple interface elements in the first user interface, the electronic device 100 may also filter out multiple interface elements that contain interactive events from the interface elements in the first user interface.

[0376] In this way, when calculating the scores of interface elements, the electronic device 100 can only calculate the scores of the multiple screened interface elements and search for the specified interface element from these screened interface elements, thereby reducing the amount of calculation of the electronic device 100 and speeding up the search speed of the electronic device 100.

[0377] Further, in some embodiments, the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface by specifying the interface element, which may specifically include: in the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, the scrolling page layout, and the Tab page layout, the electronic device 100 obtains a first element group, and the first element group includes: all child elements of the parent element of the interface element whose size is greater than a third threshold that the electronic device 100 traverses upward from the bottom layer of the ViewTree of the first user interface; when the first element group meets the layout conditions of the first pop-up window, the electronic device 100 determines the first element group as the interface element belonging to the first pop-up window in the first user interface.

[0378] Among them, evaluating the page layout of the parent element of the specified interface element can further determine whether the specified interface element searched by the electronic device 100 is the interface element in the pop-up window, thereby improving the accuracy of the pop-up window search.

[0379] Among them, whether the first element group includes the layout conditions of the first pop-up window can be determined by the electronic device 100 by calculating the score of the first element group based on the position, type, interaction event, width, and height of the interface elements contained in the first pop-up window. The score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

[0380] It can be seen that by using multiple information of interface elements in the first pop-up window to calculate the score of the first element group, multiple factors can be used to evaluate the similarity between the first element group and the pop-up window from multiple aspects, thereby improving the accuracy and comprehensiveness of pop-up window search.

[0381] For a detailed description of how the electronic device 100 searches for a pop-up window by specifying an interface element, please refer to the related contents of the aforementioned steps S601-S606.

[0382] In some embodiments, the display screen of the electronic device 100 may be a folding screen, and when the electronic device 100 displays the first pop-up window, the folding screen is in an unfolded state. In other words, the method can adjust the pop-up window to solve the problem that the pop-up window displayed by the folding screen in the unfolded state exceeds the display area of ​​the display screen, so that the adjusted pop-up window is easier for users to read and operate, thereby improving the user experience.

[0383] Fig.16A flowchart of another pop-up window adjustment method provided in an embodiment of the present application.

[0384] like Fig.16 As shown, the pop-up window adjustment method may include:

[0385] S901. The electronic device 100 displays a first pop-up window, and the size of the first pop-up window is greater than a first threshold.

[0386] Exemplarily, the first pop-up window may refer to Figure 1B The pop-up window 102 shown, or the first pop-up window may refer to Figure 2B The pop-up window 202 is shown. The first threshold may refer to the threshold 2 mentioned in step S406, that is, if the first pop-up window contains an interface element with a large screen ratio, it means that the size of the first pop-up window is too large.

[0387] S902. The electronic device 100 adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window.

[0388] For example, the first pop-up window after shrinking may refer to Figure 1C The pop-up window 102 shown, or the first pop-up window after shrinking, may refer to Figure 2C Pop-up window 202 is shown.

[0389] In some embodiments, the display screen of the electronic device 100 may be a folding screen, and when the electronic device 100 displays the first pop-up window, the folding screen is in an unfolded state. In other words, the method can address the problem of a pop-up window being too large when the folding screen is in an unfolded state, thereby reducing the pop-up window to facilitate the user to read the content in the pop-up window, while also preventing the pop-up window from being too large so that the controls in the pop-up window are too dispersed, affecting the user's control of the pop-up window, and improving the user's experience.

[0390] In addition, the first pop-up window can be displayed in the first user interface. Before adjusting the first pop-up window, the electronic device 100 can determine the interface elements in the first user interface that belong to the first pop-up window, and record the number of interface elements whose layout has changed in the first user interface, etc. For specific content not mentioned in steps S901-S902, please refer to the content in the aforementioned steps S801-S802.

[0391] Fig.17 A schematic diagram of the hardware structure of the electronic device 100 is shown.

[0392] The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0393] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0394] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0395] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0396] In some embodiments, the processor 110 can be used to perform a search for a pop-up window when the layout of the user interface changes significantly, and adjust the pop-up window after finding it. For a detailed description of the search for the pop-up window and the adjustment of the pop-up window, please refer to the above content, which will not be repeated here.

[0397] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0398] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0399] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0400] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0401] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0402] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0403] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0404] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0405] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0406] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0407] In some embodiments, the display screen 194 may be used to display a pop-up window and a user interface where the pop-up window is located. For details about the user interface, see the aforementioned Figure 1A-1E , Figure 2A-2E .

[0408] In addition, in the embodiment of the present application, if the electronic device 100 is a folding screen device, the electronic device 100 may include multiple display screens 194. Moreover, according to different opening and closing angles between the folding screens, the electronic device 100 may have different device forms.

[0409] For example, taking the electronic device 100 as a folding screen device, Fig.18 Schematic diagram of the device form of the electronic device 100 provided in the embodiment of the present application. Fig.18 As shown, electronic device 100 may include display screen 194 - 1 and display screen 194 - 2 .

[0410] in, Fig.18 (a) in the figure shows the device form of the electronic device 100 in a folded state. In the folded state, the angle between the display screen 194-1 and the display screen 194-2 is less than 180 degrees, and the electronic device 100 can display images through the display screen 194-1. Fig.18 (b) shows the device form of the electronic device 100 in the unfolded state. In the unfolded state, the angle between the display screen 194-1 and the display screen 194-2 is equal to 180 degrees, and the electronic device 100 can display the screen through the display screen 194-1 and the display screen 194-2.

[0411] Combination Figure 1A-1E and Fig.18 , Figure 1A is a user interface displayed by the electronic device 100 through the display screen 194-1, Figure 1B-Figure 1E The user interface is displayed by the electronic device 100 through the combined display screen of the display screen 194 - 1 and the display screen 194 - 2 .

[0412] Combination Figure 2A-2E and Fig.18 , Figure 2A is a user interface displayed by the electronic device 100 through the display screen 194-1, Figure 2B-2E The user interface is displayed by the electronic device 100 through the combined display screen of the display screen 194 - 1 and the display screen 194 - 2 .

[0413] It can be understood that the folding screen device provided in the embodiment of the present application may refer to an inward-folding folding screen device or an outward-folding folding screen device, wherein the inward-folding folding screen device achieves folding of the electronic device by inward folding, and the outward-folding folding screen device achieves folding of the electronic device by outward folding.

[0414] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0415] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0416] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0417] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0418] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0419] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0420] In some embodiments, the internal memory 121 can be used to store the number of interface elements that have undergone layout changes in the user interface when the electronic device 100 monitors the layout changes of the user interface. In addition, the internal memory 121 can also be used to store a whitelist, interface elements that meet preset requirements found during the pop-up window search process, element groups found based on specified interface elements or interface elements with a size greater than a threshold 2, and so on.

[0421] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0422] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0423] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0424] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0425] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0426] In some embodiments, the electronic device 100 can detect the user operation on the pop-up window through the touch sensor 180K and perform corresponding operations. For example, the electronic device 100 can detect the user operation on the close control in the pop-up window through the touch sensor 180K to trigger the closing of the pop-up window.

[0427] The electronic device may be a portable terminal device equipped with iOS, Android, Microsoft or other operating systems, such as a mobile phone, a tablet computer, a wearable device, etc., and may also be a non-portable terminal device such as a laptop computer with a touch-sensitive surface or a touch panel, a desktop computer with a touch-sensitive surface or a touch panel. The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0428] Fig.19 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0429] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0430] The application layer can include a series of application packages.

[0431] like Fig.19 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0432] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0433] like Fig.19 As shown, the application framework layer may include a pop-up window adjustment module, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0434] The pop-up window adjustment module may include: a layout monitoring module, a search module, and an adjustment module. Among them:

[0435] The layout monitoring module can be used to monitor layout changes of the user interface and record the number of interface elements in the user interface that have undergone layout changes.

[0436] The search module can be used to find pop-up windows in the user interface when the layout of the user interface changes significantly.

[0437] The adjustment module can be used to adjust the pop-up window after finding the pop-up window that needs to be adjusted in the user interface.

[0438] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0439] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0440] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0441] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0442] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0443] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0444] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0445] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0446] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0447] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0448] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0449] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0450] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0451] A 2D graphics engine is a drawing engine for 2D drawings.

[0452] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0453] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0454] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0455] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0456] The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory, so that the electronic device executes the method executed by the electronic device 100 in any of the above embodiments.

[0457] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method performed by the electronic device 100 in any of the above embodiments.

[0458] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0459] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0460] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0461] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0462] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0463] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute a method executed by any one of the electronic devices 100 in any of the above embodiments.

[0464] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes any method executed by the electronic device 100 in any of the above embodiments.

[0465] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (digitalsignal processor, DSP), an application specific integrated circuit (AP 800plication specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0466] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device executes the method in each of the above method embodiments.

[0467] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0468] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0469] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0470] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0471] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. A pop-up window adjustment method, characterized in that: The method comprises: The electronic device displays a first pop-up window, and a first interface element of the first pop-up window exceeds a display area of ​​the display screen; The electronic device adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window, or displaying a second interface element in the display area, the second interface element and the first interface element are used to trigger the same function, or moving the first interface element to the display area.

2. The method according to claim 1, characterized in that When the electronic device displays the first pop-up window, the height of the first interface element is 0, and before shrinking the first pop-up window, the method further includes: Increase the height of the first interface element.

3. A pop-up window adjustment method, characterized in that: The method comprises: The electronic device displays a first pop-up window, where a size of the first pop-up window is greater than a first threshold; The electronic device adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window.

4. The method according to any one of claims 1 to 3, characterized in that: The first pop-up window is displayed in the first user interface, and before the electronic device adjusts the first pop-up window, the method further includes: The electronic device determines the interface element in the first user interface that belongs to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface.

5. The method according to claim 4, characterized in that When the electronic device displays the first pop-up window, the number of interface elements in the first user interface whose layout changes is greater than a second threshold, Determining the interface element belonging to the first pop-up window in the first user interface specifically includes: When the number of interface elements in the first user interface that undergo layout changes is greater than a second threshold, the electronic device determines the interface elements in the first user interface that belong to the first pop-up window.

6. The method according to claim 4 or 5, characterized in that: The electronic device determines, within a preset time period, the interface elements in the first user interface that belong to the first pop-up window.

7. The method according to any one of claims 4 to 6, characterized in that: The electronic device determines, based on any one or more of the identification ID, attribute, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upwards; The electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

8. The method according to claim 7, characterized in that After the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward, the method further includes: After traversing to an interface element whose size is greater than a third threshold, the electronic device stops traversing.

9. The method according to any one of claims 4 to 8, characterized in that: The electronic device is pre-installed with a whitelist, and the whitelist includes IDs of interface elements included in one or more pop-up windows; The electronic device determines, based on one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device finds a third interface element in the first user interface that is in the whitelist; The electronic device determines the interface element belonging to the first pop-up window in the first user interface through the third interface element.

10. The method according to claim 9, characterized in that The third interface element is an interface element corresponding to the background contained in the first pop-up window, and the interface element included in the first pop-up window is a sub-element of the third interface element.

11. The method according to claims 4-8, characterized in that: The electronic device determines, based on one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device searches for a designated interface element based on the multiple pieces of information of the interface elements included in the first user interface; the designated interface element is an interface element included in the pop-up window; The electronic device determines the interface element belonging to the first pop-up window in the first user interface through the designated interface element.

12. The method according to claim 11, characterized in that The electronic device searches for a specified interface element based on the multiple pieces of information of the interface elements included in the first user interface, specifically including: The electronic device determines scores of a plurality of interface elements in the first user interface based on differences between the plurality of information of the interface elements included in the first user interface and the plurality of information of the designated interface element, wherein the score of one of the interface elements indicates a possibility that the interface element is the designated interface element; The electronic device searches for the designated interface element from the multiple interface elements based on the scores of the multiple interface elements.

13. The method according to claim 12, characterized in that The designated interface element is any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before the electronic device determines the scores of the plurality of interface elements in the first user interface, the method further includes: The electronic device filters out the multiple interface elements containing interactive events from the interface elements in the first user interface.

14. The method according to claim 13, characterized in that The electronic device determines, by using the designated interface element, an interface element in the first user interface that belongs to the first pop-up window, specifically including: In the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, the scroll page layout, and the Tab page layout, the electronic device obtains a first element group, the first element group including: all child elements of the parent element of the interface element whose size is greater than a third threshold value that the electronic device traverses upward from the bottom layer of the view tree ViewTree of the first user interface; When the first element group meets the layout condition of the first pop-up window, the electronic device determines the first element group as interface elements in the first user interface belonging to the first pop-up window.

15. The method according to claim 14, characterized in that Whether the first element group meets the layout conditions of the first pop-up window is determined by the electronic device based on the score of the first element group calculated based on the position, type, interaction event, width, and height of the interface elements contained in the first pop-up window, wherein the score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

16. The method according to any one of claims 1 to 15, characterized in that: The display screen is a folding screen, and when the electronic device displays the first pop-up window, the folding screen is in an unfolded state.

17. An electronic device, characterized in that: The electronic device comprises a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method as claimed in any one of claims 1 to 16.

18. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 16.

Images