Pop-up window adjustment method, graphical interface, and related apparatus

Abstract

Disclosed in the present application are a pop-up window adjustment method, a graphical interface, and a related apparatus. In the method, when part of content of a pop-up window exceeds a display area of a display screen, an electronic device may adjust the pop-up window, wherein the adjustment comprises: reducing the pop-up window, or newly adding an interface element in the display area, the newly added interface element and an interface element in the pop-up window which exceeds the display area being used for triggering the same function, or moving the interface element exceeding the display area into the display area. In this way, an electronic device can be prevented from displaying an incomplete pop-up window, which makes it convenient for a user to read the content in the pop-up window, or for the user to control the pop-up window, thereby improving the user experience.

Description

Pop-up window adjustment method, graphical interface and related devices

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on August 26, 2023, with application number 202311086941.7, and priority to the Chinese patent application entitled “Pop-up adjustment method, graphical interface and related devices”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminals, and in particular to a pop-up window adjustment method, a graphical interface, and related devices. Background Art

[0003] A pop-up window is a window that pops up while an application is running, used to display prompt information to users in addition to the application page.

[0004] Although pop-up windows are widely used in the terminal field, the pop-up windows of some applications are not well adapted to electronic devices, resulting in part of the content of the pop-up window exceeding the display area of ​​the display screen, and users cannot view the complete content of the pop-up window. In addition, if the content exceeding the display area includes interactive controls, such as close controls, confirmation controls, etc., users may not be able to operate the pop-up window normally, affecting the user experience.

[0005] Summary of the Invention

[0006] This application provides a pop-up window adjustment method, a graphical interface and related devices, which can adjust inappropriate pop-up windows, making the adjusted pop-up windows more suitable for user browsing and operation, and improving the user experience.

[0007] In a first aspect, an embodiment of the present application provides a pop-up window adjustment method, the method comprising: an electronic device displays a first pop-up window, and a first interface element of the first pop-up window exceeds the display area of ​​the display screen; the electronic device adjusts the first pop-up window, and the adjustment comprises: shrinking the first pop-up window, or displaying a second interface element in the display area, the second interface element and the first interface element are used to trigger the same function, or moving the first interface element into the display area.

[0008] By implementing the method provided by the first method, the electronic device can adjust the pop-up window when displaying an incomplete pop-up window so that the adjusted pop-up window can display complete content, making it easier for users to operate the pop-up window or view the complete content in the pop-up window, thereby improving the user experience.

[0009] In combination with the first aspect, in a possible implementation, when the electronic device displays the first pop-up window, the height of the first interface element is 0. Before shrinking the first pop-up window, the method further includes: increasing the height of the first interface element.

[0010] Among them, increasing the height of the interface element before shrinking the pop-up window can ensure that the first interface element can be displayed in the user interface after the pop-up window is shrunk, ensuring that the electronic device 100 can adjust the content of the pop-up window outside the display screen to within the range of the display screen.

[0011] In a second aspect, an embodiment of the present application provides a pop-up window adjustment method, the method comprising: the electronic device displays a first pop-up window, the size of the first pop-up window is greater than a first threshold; the electronic device adjusts the first pop-up window, the adjustment comprising: reducing the first pop-up window.

[0012] By implementing the method provided in the second aspect, the electronic device can reduce the size of the pop-up window when the pop-up window size is too large, so that the electronic device can display a pop-up window of appropriate size, which is convenient for users to read the content in the pop-up window. At the same time, it also avoids the pop-up window being too large, which makes the price control section in the pop-up window too scattered, affecting the user's control of the pop-up window.

[0013] In combination with the first aspect and the second aspect, in a possible implementation method, the first pop-up window is displayed in the first user interface. Before the electronic device adjusts the first pop-up window, the method also includes: the electronic device determines the interface element in the first user interface belonging to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface.

[0014] That is to say, the electronic device 100 can evaluate the information of each interface element in the user interface to find the first pop-up window from the user interface, and then successfully adjust the pop-up window.

[0015] In combination with the first aspect and the second aspect, in a possible implementation method, when the electronic device displays a first pop-up window, the number of interface elements whose layout has changed in the first user interface is greater than a second threshold, and the interface elements in the first user interface that belong to the first pop-up window are determined, specifically including: when the number of interface elements whose layout has changed in the first user interface of the electronic device is greater than the second threshold, determining the interface elements in the first user interface that belong to the first pop-up window.

[0016] In this way, the electronic device 100 can monitor the layout changes of the interface elements contained in the user interface in real time, and search for pop-up windows that need to be adjusted in the user interface based on the layout changes, so as to avoid the electronic device 100 being in a state of searching for pop-up windows all the time. The characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application are fully considered. When it is determined that a pop-up window may appear in the user interface, the search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0017] In combination with the first aspect and the second aspect, in a possible implementation manner, the electronic device determines the interface elements belonging to the first pop-up window in the first user interface within a preset time period.

[0018] In this way, the electronic device 100 can be prevented from spending too much time searching for pop-up windows, and the power consumption of the electronic device 100 can be controlled.

[0019] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, specifically including: the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward; the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

[0020] That is to say, when traversing interface elements, the electronic device 100 can traverse from the bottom layer of the ViewTree upwards. This is because if the user interface includes pop-up windows, in the ViewTree, the interface elements closer to the lower layer are drawn later and are more likely to be pop-up windows. Therefore, traversing from the bottom layer of the ViewTree upwards can minimize the number of interface elements traversed by the electronic device 100. When there are pop-up windows in the user interface, the interface elements that meet the requirements can be found as soon as possible, thereby improving the efficiency of the electronic device 100 in finding pop-up windows.

[0021] In combination with the first aspect and the second aspect, in a possible implementation, after the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface, the method also includes: the electronic device stops traversing after traversing to an interface element whose size is greater than a third threshold.

[0022] That is to say, when the electronic device 100 is traversing the interface elements, if it finds an interface element of a larger size, it can stop traversing. This is because if the user interface contains a pop-up window, when the electronic device 100 traverses to an interface element of a larger size, it means that most or all of the interface elements in the pop-up window may have been traversed. The electronic device 100 can end the traversal to avoid introducing interface elements that do not belong to the pop-up window into the calculation process during the search process, thereby saving search time.

[0023] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device is pre-installed with a whitelist, and the whitelist includes the IDs of the interface elements contained in one or more pop-up windows; the electronic device determines the interface elements in the first user interface that belong to the first pop-up window based on one or more information of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface, specifically including: the electronic device finds the third interface element in the first user interface that is in the whitelist; the electronic device determines the interface element in the first user interface that belongs to the first pop-up window through the third interface element.

[0024] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains an interface element in the whitelist.

[0025] In combination with the first aspect and the second aspect, in a possible implementation method, the third interface element is an interface element corresponding to the background contained in the first pop-up window, and the interface element included in the first pop-up window is a sub-element of the third interface element.

[0026] In combination with the first aspect and the second aspect, in a possible implementation, the third interface element is the element with the highest level in the view tree ViewTree of the first user interface among all the interface elements contained in the first pop-up window.

[0027] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface based on one or more information of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, specifically including: the electronic device searches for the specified interface element based on multiple information of the interface elements contained in the first user interface; the specified interface element is the interface element contained in the pop-up window; the electronic device determines the interface element belonging to the first pop-up window in the first user interface through the specified interface element.

[0028] That is, the electronic device 100 may first search for a landmark element in the pop-up window, and then use the landmark element to search for the pop-up window, so that the electronic device 100 can quickly find the pop-up window.

[0029] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device searches for a specified interface element based on multiple information of the interface elements contained in the first user interface, specifically including: the electronic device determines the scores of multiple interface elements in the first user interface based on the difference between the multiple information of the interface elements contained in the first user interface and the multiple information of the specified interface element, the score of an interface element indicates the possibility that the interface element is the specified interface element; the electronic device searches for the specified interface element from multiple interface elements based on the scores of the multiple interface elements.

[0030] Among them, the electronic device 100 calculates the score of the interface element through multiple information of the interface element, and can combine multiple factors to evaluate the similarity between the found interface element and the specified interface element from multiple aspects, thereby improving the accuracy and comprehensiveness of searching for the specified interface element.

[0031] In combination with the first aspect and the second aspect, in a possible implementation method, the designated interface element is any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before the electronic device determines the scores of multiple interface elements in the first user interface, the method also includes: the electronic device filters out multiple interface elements that contain interactive events from the interface elements in the first user interface.

[0032] In this way, when calculating the scores of interface elements, the electronic device 100 can only calculate the scores of the multiple filtered interface elements, and search for the specified interface element from these filtered interface elements, thereby reducing the amount of calculation of the electronic device 100 and speeding up the search speed of the electronic device 100.

[0033] In combination with the first aspect and the second aspect, in a possible implementation method, the electronic device determines the interface element belonging to the first pop-up window in the first user interface by specifying the interface element, specifically including: in the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, scrolling page layout, and Tab page layout, the electronic device obtains a first element group, and the first element group includes: all child elements of the parent element of the interface element whose size is greater than a third threshold traversed by the electronic device from the bottom layer of the view tree ViewTree of the first user interface; when the first element group meets the layout conditions of the first pop-up window, the electronic device determines the first element group as the interface element belonging to the first pop-up window in the first user interface.

[0034] Among them, evaluating the page layout of the parent element of the specified interface element can further determine whether the specified interface element searched by the electronic device 100 is the interface element in the pop-up window, thereby improving the accuracy of the pop-up window search.

[0035] In combination with the first aspect and the second aspect, in one possible implementation method, whether the first element group meets the layout conditions of the first pop-up window is determined by the electronic device based on the score of the first element group calculated based on the position, type, interaction event, width, and height of the interface elements contained in the first pop-up window, wherein the score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

[0036] It can be seen that by using multiple information of interface elements in the first pop-up window to calculate the score of the first element group, multiple factors can be used to evaluate the similarity between the first element group and the pop-up window from multiple aspects, thereby improving the accuracy and comprehensiveness of pop-up window search.

[0037] In combination with the first aspect and the second aspect, in a possible implementation, the display screen is a folding screen, and when the electronic device displays the first pop-up window, the folding screen is in an unfolded state.

[0038] In other words, this method can address the problem that the pop-up window displayed when the folding screen is in the unfolded state exceeds the display area of ​​the display screen, and adjust the pop-up window so that the adjusted pop-up window is easier for users to read and operate, thereby improving the user experience. Alternatively, this method can address the problem that the pop-up window displayed when the folding screen is in the unfolded state is too large, and then reduce the pop-up window to facilitate users to read the content in the pop-up window. At the same time, it also avoids the problem that the controls in the pop-up window are too scattered due to the pop-up window being too large, affecting the user's control of the pop-up window, thereby improving the user experience.

[0039] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method described in the first aspect or any possible implementation of the first aspect, the second aspect or any possible implementation of the second aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, cause the electronic device to execute a method as described in the first aspect or any possible implementation of the first aspect, or the second aspect or any possible implementation of the second aspect.

[0041] In a fifth aspect, a computer program product, when the computer program product is run on a computer, enables the computer to execute the method described in the first aspect or any possible implementation of the first aspect, the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figures 1A-1E and 2A-2E are some user interfaces provided in embodiments of the present application;

[0043] FIG3 is a schematic diagram of a process of monitoring the layout of a user interface by the electronic device 100 according to an embodiment of the present application;

[0044] FIG4 is a schematic diagram of the structure of a ViewTree provided in an embodiment of the present application;

[0045] Figure 5 is a flow chart of drawing interface elements;

[0046] FIG6 is an overall flow chart of searching and adjusting pop-up windows according to an embodiment of the present application;

[0047] FIG7 is a schematic diagram of the structure of a pop-up window in a ViewTree according to an embodiment of the present application;

[0048] FIG8 is a flowchart of an overall method for searching a pop-up window by using a specified interface element contained in the pop-up window provided by the electronic device 100 according to an embodiment of the present application;

[0049] FIG9 is a detailed flowchart of the electronic device 100 according to an embodiment of the present application screening out interface elements that meet the requirements from the interface elements of the user interface;

[0050] FIG10 is a schematic diagram of a process for searching for a specified interface element according to an embodiment of the present application;

[0051] FIG11 is a schematic diagram of dividing a display screen into nine areas according to an embodiment of the present application;

[0052] FIG12 is a flow chart of a method for searching for a pop-up window according to a specified interface element provided in an embodiment of the present application;

[0053] FIG13 is a schematic diagram of a process for reducing a pop-up window according to an embodiment of the present application;

[0054] FIG14 is a schematic diagram of a process of adjusting the height of a specified interface element and reducing a pop-up window by the electronic device 100 according to an embodiment of the present application;

[0055] FIG15 is a flow chart of a pop-up window adjustment method provided in an embodiment of the present application;

[0056] FIG16 is a flow chart of another pop-up window adjustment method provided in an embodiment of the present application;

[0057] FIG17 is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application;

[0058] FIG18 is a schematic diagram of the device configuration of the electronic device 100 provided in an embodiment of the present application;

[0059] FIG19 is a block diagram of the software structure of the electronic device 100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following is a clear and detailed description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0061] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0062] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0063] Pop-ups are a way for users to communicate with application products. They can be used to deliver information, status feedback, guide users, and so on.

[0064] However, the pop-up windows of some applications cannot be well adapted to electronic devices. Commonly, there is a problem that pop-up windows cannot be properly adapted on foldable screen devices. Since the foldable screen includes a folded state and an unfolded state, and the display area in the folded state is smaller than the display area in the unfolded state, if the application has a pop-up window that needs to be displayed, when the foldable screen is in the folded state, a normal-sized pop-up window can be displayed. However, when the foldable screen is in the unfolded state, since the display area in the unfolded state is larger than the display area in the folded state, the application will generally automatically enlarge the pop-up window that needs to be displayed proportionally according to the size of the display area, resulting in the device displaying a pop-up window that is too large, blocking too much content of the application page, affecting the user's perception. Furthermore, the enlarged pop-up window content is easy to exceed the display area, which not only affects the aesthetics of the interface, but more importantly, if the interactive controls in the pop-up window exceed the display screen boundary or are far away from the center of the display screen, it also affects the user's control of the pop-up window.

[0065] For details about the pop-up windows displayed in the folded state and the incompatible pop-up windows displayed in the unfolded state, please refer to the subsequent Figures 1A-1B and 2A-2B.

[0066] In order to prevent electronic devices from displaying inappropriate pop-up windows, one solution is to use third-party applications to automatically skip the pop-up windows of applications. Such third-party applications can automatically simulate click operations on pop-up windows, triggering the electronic device to automatically skip the pop-up windows of applications.

[0067] However, this has certain drawbacks:

[0068] 1) Users need to install third-party applications on their devices and authorize accessibility permissions, which makes the security of the device unprotected;

[0069] 2) When using the third-party app to skip pop-up windows, some pop-up windows require manual configuration by the user, increasing the complexity of the user operation;

[0070] 3) Third-party applications control devices to automatically skip pop-up windows, so users no longer perceive the existence of pop-up windows, which may cause users to miss important information.

[0071] An embodiment of the present application provides a pop-up window adjustment method, in which an electronic device can display a pop-up window. When the pop-up window is not suitable, the electronic device can adjust the pop-up window. The adjustment includes any one of the following: shrinking the pop-up window, or adding a new interface element, the newly added interface element and the interface element that exceeds the display area in the pop-up window are used to trigger the same function, or moving the interface element that exceeds the display area into the display area.

[0072] In some embodiments, pop-up window mismatch may occur when part of the displayed content of the pop-up window exceeds the display area of ​​the display screen. For example, part of the displayed content may include the first interface element of the first pop-up window. The content of the pop-up window that exceeds the display area may be information such as text or images in the pop-up window, which makes it impossible for the user to view the complete prompt information displayed to the user in the pop-up window. Alternatively, the content of the pop-up window that exceeds the display area may refer to interactive controls in the pop-up window, such as a close control or a confirmation control, which makes it impossible for the user to normally operate the pop-up window, thereby affecting the user's user experience.

[0073] In some embodiments, the pop-up window may not be suitable for the user because the size of the pop-up window is too large. If the size of the pop-up window is too large, the pop-up window will block most of the content of the application page, affecting the user's browsing experience.

[0074] It can be seen that when the pop-up window displayed on the electronic device is not suitable, the pop-up window adjustment method provided in the embodiment of the present application can reduce the pop-up window so that the electronic device can display a pop-up window of appropriate size. Alternatively, new interface elements for controlling the pop-up window can be added to the display area, or interface elements in the pop-up window that are outside the display area can be moved into the display area, so that the user can control the pop-up window or view the complete content in the pop-up window, and read the content in the pop-up window. At the same time, it also avoids the situation where the pop-up window is too large, which makes the controls in the pop-up window too scattered, affecting the user's control of the pop-up window.

[0075] Below, taking the electronic device 100 as a folding screen device as an example, some user interfaces provided in the embodiments of the present application are exemplarily introduced in combination with Figures 1A-1E and Figures 2A-2E.

[0076] Figures 1A and 1B are user interfaces 10 displayed on an electronic device 100 in a folded state and an unfolded state, respectively. The user interface 10 may be a user interface of application A. Application A may refer to any application installed in the electronic device 100 that has a pop-up window during display. The present embodiment of the application does not limit application A. The user interface 10 may include a home page 101 and a pop-up window 102 provided by application A. The home page 101 may be used to display multiple functions provided by application A, and the pop-up window 102 may be used to display some prompt information of the application.

[0077] It is understandable that the pop-up window 102 can be a window that pops up automatically when the electronic device 100 displays the home page 101, or it can be a window that is displayed together with the home page 101 when the electronic device 100 opens or switches to the home page 101.

[0078] If the electronic device 100 is in a folded state, as shown in FIG1A , a pop-up window 102 is used to prompt the user that a new version of the application A exists. The pop-up window 102 may include: a pop-up window main body 102A and a close control 102B. The pop-up window main body 102A may be used to display the main content of the pop-up window 102, including text, pictures and other information. For example, in the pop-up window 102 shown in FIG1A , the pop-up window main body 102A may include text information such as "Experience the new version first". The close control 102B can be used to close the pop-up window 102. Exemplarily, when the electronic device 100 detects a user operation acting on the close control 102B, such as a click operation, the electronic device 100 may close the pop-up window 102 in response to the operation.

[0079] If the electronic device 100 displays the pop-up window 102 in the expanded state, since the display area of ​​the electronic device 100 in the expanded state is larger than the display area in the folded state, the electronic device 100 will determine the size of the pop-up window 102 based on the size of the display area when displaying the pop-up window 102. For example, the electronic device 100 in the expanded state can display the pop-up window 102 as shown in Figure 1B.

[0080] By comparing Figure 1A and Figure 1B, it can be seen that since the electronic device 100 in Figure 1B is in the unfolded state, its display area is larger than the display area in Figure 1A when the electronic device 100 is in the folded state, the size of the pop-up window 102 shown in Figure 1B is larger than the pop-up window 102 shown in Figure 1A.

[0081] In a specific implementation, since the width of the display area of ​​the electronic device 100 in the expanded state is equivalent to twice the width of the display area of ​​the electronic device 100 in the folded state, the pop-up window 102 displayed by the electronic device 100 in the expanded state (see Figure 1B) is equivalent to the pop-up window 102 displayed in the folded state (see Figure 1A) after being proportionally enlarged by twice.

[0082] In addition, referring to FIG. 1B , since the size of the pop-up window 102 shown in FIG. 1B is too large, a portion of the content of the pop-up window 102 , ie, the close control 102B, exceeds the range of the display area.

[0083] In this case, the user cannot quickly close the pop-up window 102 using the close control 102B. To close the pop-up window 102, the user may need to switch the electronic device 100 from the expanded state to the folded state, so that the pop-up window 102 switches to the pop-up window 102 shown in FIG1A . In this way, the user can close the pop-up window 102 using the close control 102B only after the close control 102B of the pop-up window 102 is displayed in the display area. However, this requires the user to change the folded state of the electronic device 100, making the user operation cumbersome and reducing the user experience.

[0084] It can be understood that compared to the folded state, in the expanded user interface 10, in addition to the larger size of the pop-up window 102, the size of each interface element contained in the home page 101, or the interval between the interface elements can also change, so that the size of the home page 101 can adapt to the larger display area.

[0085] The pop-up window adjustment method provided in the embodiment of the present application can find the pop-up window that needs to be adjusted in the user interface, that is, the pop-up window that is not suitable, and adjust the pop-up window. For example, taking the pop-up window 102 shown in Figure 1B as an example, the adjustment may include:

[0086] 1) Reduce the size of the pop-up window

[0087] Exemplarily, the reduced pop-up window may refer to the pop-up window 102 shown in FIG. 1C .

[0088] In combination with Figure 1B and Figure 1C, it can be seen that after the electronic device 100 displays the user interface 10 as shown in Figure 1B, if the pop-up window 102 contained in the user interface 10 is too large, or the content in the pop-up window 102 exceeds the display area, the electronic device 100 can shrink the pop-up window 102 and display the user interface 10 shown in Figure 1C, so that the user can view all the content in the pop-up window 102, and the user can also quickly close the pop-up window 102 through the close control 102B visible in Figure 1C.

[0089] 2) Add mapping interface elements

[0090] In the case where the interface elements contained in the pop-up window exceed the display area, making it impossible for the user to control the pop-up window through the interface elements, a new interface element can be added to the display area, and the interaction events of the interface elements in the pop-up window that exceed the display area are mapped to the newly added interface element. For example, if the interface element that exceeds the display area in the pop-up window is a close control, the close control may include an interaction event, and the interaction event can be used to trigger the closing of the pop-up window based on the user's click operation. Then, when adjusting the pop-up window, a new control can be added to the pop-up window, and the click event of the original close control can be mapped to the newly added control, so that the user can trigger the closing of the pop-up window by initiating a click operation on the newly added control.

[0091] That is to say, adding a mapping interface element means adding a new interface element to the interface. The new interface element can have the same function as the interface element in the pop-up window that exceeds the display area, so that the user can control the pop-up window through the new interface element.

[0092] For example, the newly added interface element may be the close control 102C shown in FIG1D . The close control 102C may be used to close the pop-up window 102 .

[0093] 1B and 1D , it can be seen that after the electronic device 100 displays the user interface 10 shown in FIG. 1B , if the pop-up window 102 included in the user interface 10 is too large, causing the close control 102B to exceed the display area, the electronic device 100 can add a close control 102C within the visible range of the pop-up window 102. In this way, the user can quickly close the pop-up window 102 using the close control 102C.

[0094] It is understood that the electronic device 100 can place the newly added interface element in a blank area of ​​the user interface to prevent the newly added interface element from obscuring important information in the user interface. Alternatively, the electronic device 100 can place the newly added interface element in a visible blank area included in a pop-up window. The embodiment of the present application does not limit the display position of the newly added interface element.

[0095] 3) Adjust the position of interface elements

[0096] If the interface elements contained in the pop-up window exceed the display area, the interface elements exceeding the display area can be moved into the display area so that the user can view the complete content of the pop-up window.

[0097] Exemplarily, referring to FIG. 1E , the pop-up window 102 displays a close control 102B after position adjustment.

[0098] In combination with Figure 1B and Figure 1E, it can be seen that after the electronic device 100 displays the user interface 10 as shown in Figure 1B, if the close control 102B in the pop-up window 102 exceeds the display area, the electronic device 100 can move the close control 102B upward and move it to the pop-up window body 102A of the pop-up window 102, so that the user can view the complete content of the pop-up window, and the user can also close the pop-up window 102 through the moved close control 102B.

[0099] It is understood that the electronic device 100 can move the interface elements in the pop-up window that exceed the display area to the blank area of ​​the user interface to avoid blocking important information in the user interface, or further, the electronic device 100 can move the interface elements in the pop-up window that exceed the display area to the blank area visible in the pop-up window. The embodiment of the present application does not limit the display position of the interface element after adjustment.

[0100] 2A-2E below show the user interface involved in adjusting another form of pop-up window in an embodiment of the present application.

[0101] 2A and 2B are user interfaces 20 displayed in the folded and unfolded states of the electronic device 100, respectively. The user interface 20 may be a user interface of an application A installed in the electronic device 100. For a detailed description of the user interface 20, please refer to FIG. 1A , which will not be repeated here.

[0102] If the electronic device 100 is in the folded state, as shown in FIG2A , a pop-up window 202 is used to prompt the user that the application is requesting to obtain the device's location information. The pop-up window 202 may include: text message 202A, a reject control 202B, and an allow control 202C. The text message 202A includes: "The application requests to obtain your location information." The reject control 202B is used to trigger the denial of the application's permission to obtain the electronic device 100's location information. The allow control 202C is used to trigger the granting of the application's permission to obtain the electronic device 100's location information.

[0103] If the electronic device 100 displays a pop-up window 202 in the expanded state, since the display area of ​​the electronic device 100 in the expanded state is larger than the display area in the folded state, the electronic device 100 will determine the size of the pop-up window 202 based on the size of the display area when displaying the pop-up window 202. For example, the electronic device 100 in the expanded state may display a pop-up window 202 as shown in FIG. 2B , which may be a pop-up window 202 shown in FIG. 2A that is magnified twice.

[0104] In addition, referring to FIG. 2B , since the size of the pop-up window 202 shown in FIG. 2B is too large, a portion of the content of the pop-up window 202 , including the text information 202A and a portion of the permission control 202C, exceeds the range of the display area.

[0105] In this case, the user cannot view the complete text information in the pop-up window 202 , nor can the user allow the application to obtain the location information through the permission control 202C, so that the user's operation on the pop-up window 202 is limited.

[0106] Similar to the description of FIG. 1A to FIG. 1E , the pop-up window adjustment method provided in the embodiment of the present application can adjust the pop-up window 202 shown in FIG. 2B . Exemplarily, the adjustment may include:

[0107] 1) Reduce the size of the pop-up window

[0108] Exemplarily, the reduced pop-up window may refer to the pop-up window 202 shown in FIG. 2C .

[0109] 2B and 2C , it can be seen that after the electronic device 100 displays the user interface 20 shown in FIG2B , if the size of the pop-up window 202 contained in the user interface 10 is too large, or part of the content of the pop-up window 202 exceeds the display area, the electronic device 100 can shrink the pop-up window 202 and display the user interface 20 shown in FIG2C , so that the user can view all the contents in the pop-up window 202, and the user can also grant the application permission to obtain the location information of the electronic device 100 through the permission control 202C visible in FIG2C .

[0110] 2) Add mapping interface elements

[0111] The electronic device 100 can add a new interface element and map the interaction events of the interface element in the pop-up window that exceeds the display area to the newly added interface element.

[0112] For example, the newly added interface element may refer to the permission control 202D shown in Figure 2D. The permission control 202D has the same function as the permission control 202C in Figure 2A, and both are used to grant the application the permission to obtain the location information of the electronic device 100.

[0113] 2B and 2D , it can be seen that after the electronic device 100 displays the user interface 20 shown in FIG. 2B , if the pop-up window 102 included in the user interface 20 is too large, causing the permission control 202C to exceed the display area, the electronic device 100 can add a permission control 202D within the visible range of the pop-up window 202. In this way, the user can grant the application permission to obtain the location information of the electronic device 100 through the permission control 202D.

[0114] 3) Adjust the position of interface elements

[0115] For example, referring to FIG. 2E , the pop-up window 102 displays text information 202A after position adjustment, a reject control 202B, and an allow control 202C.

[0116] 2B and 2E , it can be seen that after the electronic device 100 displays the user interface 20 shown in FIG. 2B , if the text information 202A, the reject control 202B, and the allow control 202C in the pop-up window 102 exceed the display area, the electronic device 100 can move the text information 202A to the right and the reject control 202B and the allow control 202C to the left, so that the text information 202A, the reject control 202B, and the allow control 202C can be fully displayed in the display area. In this way, the user can view the complete text information in the pop-up window 202 and can also choose whether to grant the application permission to obtain the location information of the electronic device 100 by using the reject control 202B and the allow control 202C.

[0117] It is understood that for any content not mentioned in Figures 2A-2E, reference can be made to the content in Figures 1A-1E, and no further details will be given here. Furthermore, it should be noted that the pop-up windows shown in Figures 1B and 2B are oversized and their displayed content exceeds the display area. Even if the pop-up window is only oversized or its displayed content exceeds the display area, the pop-up window adjustment method proposed in this solution is still applicable.

[0118] It can be seen that for a pop-up window in the user interface, if the pop-up window is too large and / or the content of the pop-up window exceeds the display area, etc., the electronic device 100 can adjust the pop-up window so that the user can view the complete content in the pop-up window, or facilitate the user to manipulate the pop-up window.

[0119] In addition, it should be noted that the above Figures 1A to 1E and Figures 2A to 2E take the electronic device 100 as a folding screen device as an example, showing the user interface involved in the electronic device 100 adjusting the incompatible pop-up window. It should be understood that in addition to the folding screen device in the unfolded state, incompatible pop-up windows may also appear in other scenarios. For example, when a tablet displays an application through a parallel field of view, or when an application on a mobile phone is projected onto other devices (such as car-mounted devices, smart screens, etc.), the displayed pop-up window may be incompatible. In other words, the embodiment of the present application does not limit the type of device. As long as the electronic device 100 has an incompatible pop-up window that needs to be displayed, the pop-up window can be adjusted using the pop-up window adjustment method provided in the embodiment of the present application.

[0120] The following describes the principle of adjusting a pop-up window using the pop-up window adjustment method provided in an embodiment of the present application.

[0121] In general, the pop-up window adjustment method provided in the embodiment of the present application mainly includes the following two steps: 1) finding the pop-up window that needs to be adjusted in the user interface; 2) adjusting the pop-up window.

[0122] In step 1, since the pop-up window is a window that is displayed on the application page in addition to the application page, the electronic device 100 can monitor the layout changes of the interface elements included in the user interface in real time and search for the pop-up window that needs to be adjusted in the user interface based on the layout changes. For example, if it is detected that the layout changes of the interface elements included in the user interface are large, it means that there may be a pop-up window that is newly displayed on the application page, and a search for the pop-up window can be triggered.

[0123] FIG3 is a schematic diagram showing a flow chart of the electronic device 100 monitoring the layout of the user interface.

[0124] S101. The electronic device 100 records the number of interface elements in the user interface whose layout has changed.

[0125] The electronic device 100 may record in real time the number of interface elements whose layouts have changed in the user interface during the process of displaying the user interface. In the embodiment of the present application, the user interface displayed by the electronic device 100 may also be referred to as a first user interface.

[0126] Interface elements (View or ViewGroup) are elements that make up a user interface. A user interface is composed of multiple interface elements, including TextView, ImageView, ButtonView, and so on. For example, referring to Figure 2A, text message 202A may refer to a TextView, while the reject control 202B and the allow control 202C may refer to ButtonView.

[0127] The layout change of the interface elements includes changes in one or more of the display effects such as the position, size, color, etc. of the interface elements.

[0128] To better understand the relationship between interface elements and user interface, the following takes the Android system as an example to introduce the relationship between user interface and interface elements.

[0129] When drawing a user interface, the electronic device 100 will first obtain the layout file corresponding to the user interface, which records the interface elements contained in the user interface and the association relationship between these interface elements in the user interface. This association relationship is reflected by the view tree ViewTree corresponding to the user interface. ViewTree is a tree structure organized by interface elements in the Android user interface framework. In the ViewTree, an interface element (ViewGroup) can include child nodes (View or ViewGroup), and an interface element (View or ViewGroup) can also include a parent node (ViewGroup).

[0130] FIG4 exemplarily shows a schematic structural diagram of a ViewTree.

[0131] As shown in Figure 4, the ViewTree is composed of interface elements such as DecorView, ViewGroup, and View. Among them, DecorView is the root node of ViewTree, the highest level in ViewTree, and covers the entire display area of ​​the display screen. ViewGroup is the parent node of an interface element, and View is the child node of an interface element.

[0132] As can be seen from Figure 4, the root node DecorView includes two child nodes (ViewGroup1, View2), ViewGroup1 includes two child nodes (View3, ViewGroup4), ViewGroup4 includes one child node (View5), and ViewGroup1 and View2 are sibling elements, and View3 and ViewGroup4 are sibling elements.

[0133] Before the electronic device 100 displays a user interface, the electronic device 100 draws each interface element layer by layer starting from the root node through the ViewTree corresponding to the user interface.

[0134] As shown in Figure 5, the drawing of an interface element is completed by sequentially executing the three steps of measure, layout, and draw. Measure is used to measure the width and height of a View, layout is used to position a View within a ViewGroup, and draw is used to draw an image on the View. Furthermore, drawing an interface element is a recursive process: the parent element draws itself and its children, and then the children draw themselves and their children.

[0135] In an embodiment of the present application, the electronic device 100 can record the interface elements that have undergone layout changes in the user interface during the Layout process. In a specific implementation, the electronic device 100 can perform plug-in operations during the Layout process to monitor the layout changes of the interface elements and save the information of the interface elements that have undergone layout changes in each frame to a collection A. In this way, before drawing the next frame of the image, the number of interface elements saved in the collection A is the number of interface elements that have undergone layout changes in the current user interface in the previous frame.

[0136] S102. The electronic device 100 determines whether the number is greater than a threshold value 1.

[0137] Exemplarily, the threshold 1 may be 50 or 80, etc., and the embodiment of the present application does not limit the threshold 1.

[0138] If the number is greater than the threshold value of 1, it indicates that the layout of the user interface drawn by the electronic device 100 has changed significantly, possibly due to page switching, the addition of new pop-up windows, etc. In this case, the electronic device 100 can search for pop-up windows in the user interface, i.e., execute step S103 to determine whether there are pop-up windows in the user interface that need to be adjusted. Otherwise, the electronic device 100 can record and monitor changes in the layout of the user interface.

[0139] In the embodiment of the present application, the threshold 1 may also be referred to as a second threshold.

[0140] S103. The electronic device 100 searches for pop-up windows in the user interface, and adjusts the pop-up window after finding the pop-up window that needs to be adjusted.

[0141] When the layout of the user interface changes significantly, the electronic device 100 can perform a search for pop-up windows in the user interface, that is, find the pop-up windows that need to be adjusted from the user interface, which may include: traversing each interface element in the user interface, and finding pop-up windows from the interface elements contained in the user interface based on the layout of the position, size, etc. of the interface elements.

[0142] For a detailed description of step S103, please refer to the contents of subsequent Figures 6 to 14, which will not be expanded here.

[0143] S104. The electronic device 100 determines whether the search time exceeds a preset time.

[0144] The search duration refers to the time it takes for the electronic device 100 to search for the pop-up window. For example, the preset duration is 30ms or 50ms, etc. The embodiment of the present application does not limit the preset duration.

[0145] In a specific implementation, after the electronic device 100 executes step S102, if the number is greater than the threshold 1, the electronic device 100 can start the timer and, during the execution of step S103, determine whether the execution duration of step S103 exceeds the preset duration set by the timer.

[0146] If the execution time of step S103 exceeds the preset time, that is, the electronic device 100 searches for the pop-up window in the user interface for a period of time exceeding the preset time, the electronic device 100 executes step S105 to end the search for the pop-up window. Otherwise, the electronic device 100 may execute step S103 to continue searching for the pop-up window in the user interface.

[0147] In this way, the electronic device 100 can be prevented from spending too much time searching for pop-up windows, and the power consumption of the electronic device 100 can be controlled.

[0148] It is understood that step S104 is an optional step, and step S105 may be performed after the electronic device 100 performs step S103. In other words, the electronic device 100 may end the search for the pop-up window after completing the search for the pop-up window. The completion of the search for the pop-up window here includes two situations: the electronic device 100 finds the pop-up window in the user interface, and the electronic device 100 determines that the user interface does not contain the pop-up window that needs to be adjusted.

[0149] S105. The electronic device 100 ends searching for the pop-up window in the user interface.

[0150] Considering that the drawing of interface elements includes the three processes of measure, layout, and draw, the execution timing of steps S101-S105 can exist in the following two situations:

[0151] 1) In this embodiment of the application, steps S101-S105 can be performed after the Layout process and before the Draw process.

[0152] In other words, the electronic device 100 can start monitoring the layout of the user interface before drawing the user interface each time, and when the layout changes significantly, perform a search for pop-up windows and adjust the pop-up windows that need to be adjusted. Since the draw process is used to draw images on interface elements to display interface elements, steps S101-S105 are executed after the Layout process and before the draw process. This allows the electronic device 100 to find the pop-up window before displaying it in the user interface, and determine in advance whether the pop-up window to be displayed is the pop-up window that needs to be adjusted, and adjust the pop-up window. In this way, when the electronic device 100 displays a pop-up window, the adjusted pop-up window can be displayed directly without displaying the pop-up window before adjustment. In this way, the user does not need to view pop-up windows of inappropriate size.

[0153] Taking the pop-up window 102 shown in Figure 1B as an example, when the electronic device 100 needs to display the pop-up window 102 shown in Figure 1B, it can adjust the pop-up window 102 before displaying the pop-up window 102. In this way, the electronic device 100 can directly display the pop-up window 102 shown in Figure 1C or Figure 1D or Figure 1E, instead of displaying the inappropriate pop-up window 102 shown in Figure 1B.

[0154] Taking the pop-up window 202 shown in Figure 2B as an example, when the electronic device 100 needs to display the pop-up window 202 shown in Figure 2B, it can adjust the pop-up window 202 before displaying the pop-up window 202. In this way, the electronic device 100 can directly display the pop-up window 102 shown in Figure 2C or Figure 2D or Figure 2E, instead of displaying the inappropriate pop-up window 202 shown in Figure 2B.

[0155] 2) In this embodiment of the application, steps S101-S105 may be executed after the draw process.

[0156] In other words, the electronic device 100 can start monitoring the layout of the user interface after each drawing, and when the layout changes significantly, perform a search for pop-up windows and adjust the pop-up windows that need to be adjusted. In this case, the electronic device 100 can adjust the pop-up window after displaying the pop-up window that needs to be adjusted in the user interface. In this way, the electronic device 100 can display the pop-up window before and after the adjustment, and the user can view the dynamic adjustment process of the pop-up window.

[0157] Taking the pop-up window 102 shown in FIG. 1B as an example, after displaying the pop-up window 102 shown in FIG. 1B , the electronic device 100 adjusts the pop-up window 102 , and then displays the pop-up window 102 shown in FIG. 1C , FIG. 1D , or FIG. 1E .

[0158] Taking the pop-up window 202 shown in FIG. 2B as an example, after displaying the pop-up window 202 shown in FIG. 2B , the electronic device 100 adjusts the pop-up window 202 , and then displays the pop-up window 202 shown in FIG. 2C , FIG. 2D , or FIG. 2E .

[0159] In a specific implementation, the electronic device 100 may perform function insertion between the Layout process and the draw process, or after the draw process, and the inserted function is used to implement steps S101-S105.

[0160] In general, monitoring the layout changes in the user interface and performing a search for pop-up windows when the layout changes significantly can avoid the electronic device 100 being in a state of searching for pop-up windows all the time. It fully considers the characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application. When it is determined that a pop-up window may appear in the user interface, a search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0161] The following describes the details of step S103 in conjunction with FIG. 6 to FIG. 14 .

[0162] FIG6 is an overall flow chart of searching and adjusting pop-up windows provided in an embodiment of the present application.

[0163] As shown in FIG6 , the steps of searching and adjusting the pop-up window mentioned in step S103 above mainly include the following:

[0164] S201. The electronic device 100 searches the user interface for an interface element belonging to a pop-up window that needs to be adjusted.

[0165] In a specific implementation, the electronic device 100 can obtain all the interface elements contained in the user interface through the layout file of the user interface. The layout file records the interface elements contained in the interface element and the association relationship between these interface elements in the user interface. The association relationship refers to the parent-child relationship, sibling relationship, etc. between nodes in the ViewTree. For a specific description of ViewTree, please refer to the relevant content of Figure 4 above, which will not be repeated here.

[0166] If a pop-up window is displayed in the user interface, the interface elements included in the user interface include one or more interface elements corresponding to the pop-up window.

[0167] Exemplarily, taking the pop-up window 102 shown in FIG. 1A and FIG. 1B as an example, FIG. 7 shows a schematic structural diagram of the pop-up window in the ViewTree.

[0168] As shown in FIG7 , the pop-up window 102 may be composed of three interface elements: ViewGroup6, View7, and View8. ViewGroup6 may refer to the background of the pop-up window 102, View7 may refer to the pop-up window main body 102A in the pop-up window 102, and View8 may refer to the close control 102B. ViewGroup6 is the parent element of View7 and View8.

[0169] It should be noted that the background of the pop-up window 102 can be an invisible transparent background or a visible translucent background, such as a gray translucent background. In addition, the background of the pop-up window 102 is shown in the form of a dotted box in Figure 7, which does not mean that the background of the pop-up window 102 is the range where the dotted box is located. For example, the range where the background of the pop-up window 102 is located can be the range where the display screen is located.

[0170] The pop-up window that needs to be adjusted is an unsuitable pop-up window, which may be a pop-up window that is too large and / or a pop-up window whose content exceeds the display area. Therefore, in the process of searching for pop-ups, the interface element in the user interface that is the pop-up window that needs to be adjusted can be determined based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the user interface.

[0171] For a detailed description of the electronic device search pop-up window, please refer to the subsequent content and will not be expanded here.

[0172] S202. The electronic device 100 adjusts the found pop-up window.

[0173] In the embodiment of the present application, the found pop-up window can also be referred to as the first pop-up window.

[0174] Among them, if the size of the pop-up window is too large, the electronic device 100 displays a pop-up window with a size greater than the threshold in the user interface before adjustment, and the adjustment may refer to reducing the pop-up window. If a portion of the content of the pop-up window exceeds the display area, the electronic device 100 displays a portion of the content of the pop-up window in the user interface before adjustment, and another portion of the content, such as the first interface element, exceeds the display area of ​​the display screen. The adjustment may refer to reducing the pop-up window, or adding a new interface element in the display area, and the newly added interface element is used to trigger the same function as the interface element in the pop-up window that exceeds the display area, or moving the interface element that exceeds the display area into the display area.

[0175] For example, the electronic device 100 may reduce the size of the pop-up window by any of the following methods:

[0176] 1) Reduce the pop-up window according to the specified reduction ratio

[0177] For example, the electronic device 100 may reduce the size of the pop-up window to 70% of its original size. This embodiment of the present application does not impose any limitation on the reduction ratio.

[0178] 2) Shrink the pop-up window until all its contents are within the display area.

[0179] In this case, the reduction ratio of the pop-up window is not fixed, and the electronic device 100 can adaptively adjust the reduction ratio of the pop-up window according to the size of the pop-up window.

[0180] It is understandable that the embodiments of the present application do not limit the method of reducing the pop-up window.

[0181] It can be seen that reducing the size of the pop-up window can prevent the pop-up window from blocking too much content on the application page. Moreover, if the pop-up window displays content that exceeds the display area, reducing the size of the pop-up window can also make the entire content of the pop-up window as much as possible within the display area, making it easier for users to view the entire content of the pop-up window.

[0182] If the content in the pop-up window that exceeds the display area includes a first interface element, then a newly added interface element in the display area, such as a second interface element, can be used to trigger the same function as the first interface element. For example, taking the first interface element as a close control, the newly added second interface element and the first interface element can both be used to trigger the electronic device 100 to close the pop-up window. In a specific implementation, the electronic device 100 can map the interaction event of the first interface element to the second interface element.

[0183] It can be seen that mapping the interactive events of the second interface element to the first interface element can ensure that the pop-up window located in the display area can contain complete interactive events as much as possible. In this way, even if the current pop-up window is not displayed completely, the pop-up window still contains complete functions, which is convenient for users to operate the pop-up window.

[0184] If the content in the pop-up window that exceeds the display area includes a first interface element, moving the interface element that exceeds the display area into the display area may refer to moving the first interface element into the display area. The first interface element may be text, a picture, a control, or the like. The electronic device 100 may move the first interface element to any position in the display area. Preferably, the first interface element may be moved to a blank area of ​​the pop-up window. This not only prevents the first interface element from blocking other content, but also allows the user to see that the first interface element belongs to the content in the pop-up window.

[0185] It can be seen that moving the interface elements that exceed the display area into the display area can ensure that the user can view the complete content in the pop-up window as much as possible, and also makes it easier for the user to manipulate the pop-up window.

[0186] Taking the pop-up window 102 shown in Figure 1B as an example, the reduced pop-up window may refer to the pop-up window 102 shown in Figure 1C, the first interface element may refer to the close control 102B shown in Figure 1B (see the close control 102B shown in Figure 1A), the second interface element may refer to the close control 102C shown in Figure 1D, and the first interface element after moving may refer to the close control 102B shown in Figure 1E.

[0187] Taking the pop-up window 202 shown in Figure 2B as an example, the reduced pop-up window may refer to the pop-up window 202 shown in Figure 1C, the first interface element may refer to the allow control 202C shown in Figure 2B (see the allow control 202C shown in Figure 2A), the second interface element may refer to the allow control 202D shown in Figure 2D, the first interface element before movement can refer to the text message 202A, the reject control 202B, and the allow control 202C shown in Figure 2B, and the first interface element after movement can refer to the text message 202A, the reject control 202B, and the allow control 202C shown in Figure 2E.

[0188] In the embodiment of the present application, the search pop-up window mentioned in step S201 may include the following two methods:

[0189] 1) Find pop-ups by searching for whitelisted elements in the user interface

[0190] The whitelist is a configuration library created and maintained by developers. It includes information about interface elements contained in one or more pop-up windows, which can be the element identifiers (IDs) of the interface elements. Developers can generate the whitelist by collecting information about interface elements contained in pop-up windows that need to be adjusted, displayed in different applications.

[0191] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains interface elements on the whitelist.

[0192] The interface element on the whitelist may refer to the element with the highest level in the ViewTree among all the interface elements contained in the pop-up window. Referring to FIG7 , taking pop-up window 102 as an example, the element with the highest level may refer to ViewGroup 6. Then, after the electronic device 100 finds the interface element in the pop-up window from the user interface using the whitelist, it may determine all sub-elements of the found interface element as the interface element contained in the pop-up window.

[0193] 2) Find the pop-up window by specifying the interface elements contained in the pop-up window

[0194] Because pop-ups differ from app pages, they typically have certain fixed features that distinguish them from app pages. For example, pop-ups often contain designated interface elements such as close controls, confirm controls, cancel controls, reject controls, and allow controls. These designated interface elements are also relatively fixed in position. For example, a close control is typically located in the lower center or upper right corner of the display, allowing users to manipulate the pop-up window through these designated interface elements.

[0195] It is also understandable that the designated interface element is a necessary interface element in the pop-up window. If the designated interface element exceeds the display area or exceeds the area convenient for the user to manipulate, it may affect the user's manipulation of the pop-up window or affect the user's reading of the prompt information displayed in the pop-up window. Therefore, when searching for a pop-up window by the designated interface element, if the designated interface element exceeds the display area or exceeds the area convenient for the user to manipulate, if it is determined that the designated interface element is an element in the pop-up window, then the pop-up window can be determined to be a pop-up window that needs to be adjusted.

[0196] Therefore, when the electronic device 100 searches for a pop-up window, it can first search for a designated interface element in the user interface that may belong to a pop-up window, and then search for the pop-up window through the designated interface element.

[0197] For the detailed process of method 2, please refer to the relevant content of Figure 8, which will not be expanded here.

[0198] In some embodiments, the electronic device 100 may combine the two aforementioned methods to search for pop-up windows. For example, the electronic device 100 may first use method 1 to search whether the user interface contains an interface element on the whitelist. If not, the electronic device 100 may then use method 2 to search whether the user interface contains a specified interface element, and then search for a pop-up window using the specified interface element found.

[0199] In addition, the information of the interface elements in each pop-up window can be continuously collected, and the white list can be updated regularly so that the electronic device 100 can directly use the white list to more accurately identify the pop-up window. Among them, the information of the collected interface elements can come from the interface elements in the pop-up window found by each electronic device using the above method 2, or from the pop-up window extracted by the developer when testing each application. For example, after each electronic device finds the pop-up window using the above method 2, it can send the interface elements contained in the pop-up window to the server in a centralized manner, and the server collects the interface elements in the pop-up window identified by each device, and then updates these interface elements to the white list, and re-sends the updated white list to each electronic device, so that each electronic device can directly use method 1 to identify the pop-up window, and minimize the number of times the electronic device uses method 2 to identify the pop-up window, thereby reducing the amount of computing power of the electronic device.

[0200] FIG8 shows a flowchart of an overall method for the electronic device 100 to search for a pop-up window through a specified interface element contained in the pop-up window.

[0201] As shown in Figure 8, searching for a pop-up window through a specified interface element contained in the pop-up window mainly includes:

[0202] S301. The electronic device 100 searches for a specified interface element in a pop-up window from the interface elements included in the user interface.

[0203] Exemplarily, the designated interface element may refer to a landmark element in a pop-up window such as a close control, a confirmation control, a cancel control, a reject control, or an allow control.

[0204] The electronic device 100 can find the specified interface element from the interface elements included in the user interface based on multiple factors included in the specified interface element, such as ID, attributes, width, height, position, interaction event, etc.

[0205] For example, if an interface element is designated as a close control, the electronic device 100 may search for a close control from all interface elements in the user interface according to one or more of the following requirements: the element ID contains the keyword "close", the interface element is located far from the center of the display screen, the interface element contains an interactive event, etc. Among all interface elements, the one that meets the above requirements is the close control.

[0206] S302. The electronic device 100 searches for a pop-up window that needs to be adjusted and is included in the user interface according to the designated interface element.

[0207] After the electronic device 100 finds the specified interface element that may belong to the pop-up window, the electronic device 100 can continue to search for interface elements that are associated with the specified interface element through the specified interface element, and then determine whether these elements are the interface elements included in the pop-up window based on the layout between these associated interface elements, thereby realizing the search for the pop-up window.

[0208] The specific description of step S302 can be found in the subsequent FIG12 , which will not be expanded here.

[0209] Among them, in step S301, since the user interface contains a large number of interface elements, in order to speed up the speed of the electronic device 100 to find the specified interface element, when searching for the specified interface element, the electronic device 100 can first filter out the interface elements that preliminarily meet the requirements of the specified interface element from the interface elements of the user interface, and then find the specified interface element from these interface elements that meet the requirements.

[0210] That is, in some implementations, step S301 may include two stages:

[0211] Phase 1: Filter out interface elements that meet preset requirements from the interface elements of the user interface

[0212] The preset requirement may refer to a requirement that all specified interface elements contained in any pop-up window must meet.

[0213] For example, if the specified interface element is any one of a close control, a confirmation control, a cancel control, a reject control, or an allow control, the requirement may mean that the interface element includes an interaction event. This is because these interface elements may have different positions and forms within a pop-up window, but they all include an interaction event. The interaction event is used to enable the electronic device 100 to trigger a certain function based on the user operation on the close control, for example, the close control is used to trigger the closing of the pop-up window.

[0214] Therefore, the electronic device 100 can first filter out a part of the interface elements according to certain requirements, and then search for the specified interface element from this part of the filtered interface elements. In this way, the computing power of the electronic device 100 can be reduced and the search speed of the electronic device 100 can be accelerated.

[0215] In a specific embodiment, FIG9 shows a detailed flow chart of the electronic device 100 screening out interface elements that meet the requirements from the interface elements of the user interface.

[0216] As shown in FIG9 , stage 1 may include the following steps:

[0217] S401. The electronic device 100 traverses the interface elements in the user interface from the bottom layer of the ViewTree of the user interface upwards.

[0218] The ViewTree of the user interface records all the interface elements contained in the user interface. The electronic device 100 can traverse the interface elements in the user interface from the bottom layer of the ViewTree of the user interface, and then determine the interface elements that belong to the pop-up window in the first user interface based on any one or more of the above information of the traversed interface elements.

[0219] This is because if the user interface includes a pop-up window, in the ViewTree, the interface elements closer to the lower layer are drawn later and are more likely to be pop-ups. Therefore, traversing upward from the bottom layer of the ViewTree can minimize the number of interface elements traversed by the electronic device 100, and when there is a pop-up window in the user interface, the interface element that meets the requirements can be found as soon as possible.

[0220] It is understandable that the electronic device 100 may also start from the root node at the top level of the ViewTree and traverse downwards, which is not limited in this embodiment of the present application.

[0221] S402. The electronic device 100 determines whether the currently traversed interface element is in the whitelist.

[0222] The whitelist can be a configuration library pre-set by the developer in the electronic device 100. The whitelist can be used to record the highest-level element in the ViewTree among all the interface elements contained in the pop-up window. For example, the electronic device 100 can determine whether the currently traversed interface element is the interface element in the pop-up window by determining whether the element ID of the currently traversed interface element is in the whitelist.

[0223] If the currently traversed interface element is in the whitelist, the electronic device 100 executes step S403 , otherwise, executes step S404 .

[0224] S403. The electronic device 100 ends the traversal and determines all sub-elements of the currently traversed interface element as the interface elements included in the pop-up window.

[0225] If the interface element currently traversed by the electronic device 100 (for example, the third interface element) is in the whitelist, it means that the user interface contains a pop-up window, and the currently traversed interface element is the highest-level element in the ViewTree contained in the pop-up window.

[0226] Exemplarily, taking the pop-up window shown in FIG. 7 as an example, the currently traversed interface element may refer to ViewGroup6. View7 and View8, as child elements of ViewGroup6, may be determined as the interface elements contained in the pop-up window.

[0227] It is understandable that steps S402 and S403 are optional steps. If the electronic device 100 does not have a preset whitelist, the electronic device 100 can directly search for interface elements in the user interface that meet the preset requirements, that is, execute step S404.

[0228] S404. The electronic device 100 determines whether the currently traversed interface element meets the preset requirements.

[0229] If the interface element currently traversed by the electronic device 100 is not in the whitelist, the electronic device 100 may further determine whether the interface element meets the preset requirements.

[0230] Among them, the preset requirement may mean that the interface element contains an interactive event, that is, the electronic device 100 can detect the user operation acting on the interface element and trigger a corresponding response.

[0231] If the currently traversed interface element meets the preset requirements, the electronic device 100 executes step S405 to save the interface element to set B. Otherwise, the electronic device 100 executes step S406 to further determine whether the screen ratio of the currently traversed interface element is greater than the threshold.

[0232] S405. The electronic device 100 saves the interface element into set B.

[0233] The electronic device 100 saves the interface element into set B, so that set B can collect interface elements in the user interface that meet preset requirements.

[0234] S406. The electronic device 100 determines whether the screen ratio of the currently traversed interface element is greater than a threshold value 2.

[0235] Exemplarily, the threshold 2 may be 60%-80%, and the embodiment of the present application does not limit the threshold 2.

[0236] If the screen ratio of the currently traversed interface element is greater than the threshold 2, the electronic device 100 executes step S407 to end the traversal; otherwise, the electronic device 100 executes step S401 to continue traversing the next interface element.

[0237] That is to say, when the electronic device 100 is searching for an interface element that meets the preset requirements in the user interface, if an interface element with a larger size is found, the traversal is terminated.

[0238] In the embodiment of the present application, the threshold 2 may also be referred to as a third threshold.

[0239] For example, taking the pop-up window shown in FIG. 7 as an example, the interface element with a larger size may refer to View7.

[0240] In this way, if the electronic device 100 traverses from the bottom layer of the ViewTree upwards, when the electronic device 100 finds an interface element with a larger size, it means that the electronic device 100 may have traversed most or all of the interface elements in the pop-up window. The electronic device 100 can end the traversal to avoid saving the interface elements that do not belong to the pop-up window into the set B, saving search time.

[0241] S407. The electronic device 100 ends the traversal.

[0242] It can be seen from steps S401 to S407 that the electronic device 100 can find one or more interface elements that meet preset requirements by traversing the interface elements in the user interface.

[0243] Phase 2: Find the specified interface elements in the pop-up window from the interface elements that meet the preset requirements

[0244] Since multiple interface elements that meet the preset requirements may be found in stage one, the electronic device 100 can further combine other possible features of the specified interface element to find the element that best meets the characteristics of the specified interface element from these multiple interface elements that meet the preset requirements.

[0245] For example, FIG10 is a flow chart of searching for a specified interface element provided in an embodiment of the present application.

[0246] As shown in Figure 10, stage 2 may include the following steps:

[0247] S501. The electronic device 100 determines the scores of the interface elements that meet the preset requirements.

[0248] The electronic device 100 may use a scoring rule to determine a score of an interface element that meets preset requirements, where the score indicates the possibility that the interface element is a designated interface element.

[0249] In the scoring system, multiple rules can be established, which describe the characteristics that the specified interface element may have, and the characteristics that it does not have. The score of the interface element can be determined by comparing whether the interface element meets these characteristics and evaluating the differences between these interface elements and the specified layout elements in terms of element ID, attributes, width, height, position, etc. Among them, if the interface element contains the characteristics that the specified interface element may have, then the interface element will be given extra points, and if the interface element contains the characteristics that the specified interface element may not have, then the interface element will be deducted points.

[0250] In this way, the scoring rule can be used to determine the likelihood that each interface element that meets the preset requirements is the designated interface element, thereby achieving the goal of finding the designated interface element from these interface elements that meet the preset requirements. Furthermore, using the scoring rule to find the designated interface element can combine multiple factors and evaluate the similarity between the found interface element and the designated interface element from multiple aspects, thereby improving the accuracy of finding the designated interface element.

[0251] S502. The electronic device 100 determines whether the interface elements that meet the preset requirements include an interface element with a score greater than a threshold value 3.

[0252] Exemplarily, the threshold 3 may be 0. The embodiment of the present application does not limit the threshold 3.

[0253] If the interface elements that meet the preset requirements include an interface element with a score greater than the threshold 3, the electronic device 100 executes step S503; otherwise, it executes step S504.

[0254] It is understood that step S502 is an optional step. After determining the scores of each interface element, the electronic device 100 can directly determine the interface element with the highest score as the designated interface element. Step S502 can improve the evaluation criteria for designated interface elements and improve the accuracy of finding designated interface elements.

[0255] S503. The electronic device 100 determines the interface element with the highest score as the designated interface element.

[0256] The interface element with the highest score indicates that the interface element is most likely to be the designated interface element. In other words, the electronic device 100 can determine the interface element with the highest score and a score greater than a threshold of 3 as the designated interface element.

[0257] S504. The electronic device 100 determines that the user interface does not include a pop-up window that needs to be adjusted.

[0258] If the interface elements meeting the preset requirements do not include an interface element with a score greater than the threshold 3, it means that the interface elements meeting the preset requirements do not include the specified interface element, that is, the user interface does not include the pop-up window that needs to be adjusted.

[0259] In this way, the electronic device 100 can end the search for pop-up windows and avoid mistaking elements similar to the specified interface elements as the specified interface elements as much as possible, thereby saving the electronic device 100 from subsequent invalid operations of searching for pop-up windows based on the specified interface elements.

[0260] It is understandable that the electronic device 100 may also skip the process of screening layout elements that meet preset requirements in stage 1, and directly execute stage 2 to calculate the scores of each interface element in the user interface, and find the specified interface element according to the score.

[0261] To better understand steps S501-S504, the following takes specifying an interface element as a close control as an example to describe a specific example of how the electronic device 100 uses a scoring rule to find a close control from multiple interface elements.

[0262] For example, Table 1 shows a scoring rule table for finding a close control provided in an embodiment of the present application.

[0263] Table 1

[0264] Among them, in the above Table 1:

[0265] (1) The element attribute value in Rule 2 can be obtained through the getContentDescription() method, which can obtain the contentDescription attribute value of the interface element. Among them, this attribute value is mainly used in accessibility services to provide additional explanations of controls for people with visual impairments. For example, when a user clicks on a control through accessibility services, the electronic device 100 can extract the contentDescription attribute value of the control through the getContentDescription() method and play it in voice, so that the user can determine the control clicked by listening to the voice.

[0266] (2) If the interface element is a TextView, the interface element itself has a Text attribute, which can carry some explanatory text labels.

[0267] It is understandable that if the interface element is a TextView, Rule 3 can be used to determine whether the interface element is related to keywords such as "close", "give up", "skip", "reject", etc., without having to repeat the judgment through Rule 2. If the interface element is not a TextView, Rule 2 can be used to determine whether the interface element is related to keywords such as "close", "give up", "skip", "reject", etc. In this way, the amount of calculation of the electronic device 100 can be reduced. Therefore, when using the scoring rule table shown in Table 1 above to determine the score of the interface element, in addition to using Rule 1 and Rules 4-6 to determine the score of the interface element, you can choose one of Rule 2 and Rule 3 to calculate the score of the interface element.

[0268] (3) In rule 4, the width or height of the interface element is larger, which may mean that the width or height of the interface element is larger than threshold 4. For example, threshold 4 may be 350px or 400px. This embodiment of the application does not limit threshold 4.

[0269] (4) The aspect ratio or height-to-width ratio of the interface element in rule 5 is large, which may mean that the aspect ratio or height-to-width ratio of the interface element is greater than a threshold value 5. For example, the threshold value 5 may be 1.3. The embodiment of the present application does not impose any limitation on the threshold value 5.

[0270] (5) In Rule 6, since the close control of the pop-up window is usually located in the lower middle or upper right corner of the display screen, the possibility that the interface element is a close control can be determined based on the position of the interface element in the display screen.

[0271] For example, the display screen may be divided into nine 3*3 areas. FIG11 shows a schematic diagram of dividing the display screen into nine areas.

[0272] As shown in Figure 11, the display screen may include nine areas: Area 1 through Area 9. Area 3 is the upper right corner of the display screen, and Area 8 is the lower center area of ​​the display screen. Rule 6 may mean that interface elements located in Area 3 or Area 8 are awarded points, and that the further away from the center point of Area 5, the more points are awarded.

[0273] In addition, it should be noted that the points subtracted by Rule 4 or Rule 5 can change dynamically. For example, in Rule 4, if the width or height of the interface element is greater than the threshold 4' and less than the threshold 4", 1 point will be subtracted; if the width or height of the interface element is greater than the threshold 4", 2 points will be subtracted, and so on. In other words, the larger the width or height of the interface element, the more points will be subtracted. For another example, in Rule 5, if the aspect ratio or height-to-width ratio of the interface element is greater than the threshold 5' and less than the threshold 5", 1 point will be subtracted; if the width or height of the interface element is greater than the threshold 5", 2 points will be subtracted, and so on. In other words, the larger the aspect ratio or height-to-width ratio of the interface element, the more points will be subtracted.

[0274] It can be understood that Table 1 is only an illustrative introduction to the example application of the scoring system rules in searching for specified interface elements. In the embodiment of the present application, when searching for specified interface elements, the scoring system rules can also include more or fewer rules, and the rules set in the scoring system rules can also be different for different specified interface elements to be searched, and the embodiment of the present application does not limit this.

[0275] After finding the designated interface element, the electronic device 100 may search for the pop-up window that needs to be adjusted contained in the user interface according to the designated interface element.

[0276] FIG12 is a flow chart of a method for searching for a pop-up window based on a specified interface element provided in an embodiment of the present application.

[0277] As shown in FIG12 , searching for a pop-up window according to a specified layout space may include the following steps:

[0278] S601. The electronic device 100 searches for a parent element of a specified interface element in the user interface.

[0279] For example, the electronic device 100 may search for the parent element of the specified interface element from the ViewTree corresponding to the user interface. The specified interface element may have multiple parent elements. The parent element may be a direct parent element or an indirect parent element. The electronic device 100 may search for the parent element of the specified interface element by traversing the ViewTree.

[0280] Taking Figure 4 as an example, assuming that View5 is a designated interface element, the parent elements of View5 may include ViewGroup4, ViewGroup1, and DecorView, wherein ViewGroup4 is the direct parent element of View5, and ViewGroup1 and DecorView are indirect parent elements of View5.

[0281] S602. The electronic device 100 determines whether the parent element of the specified interface element includes an interface element that does not meet the conditions.

[0282] Among them, it can be determined whether all parent elements of the specified interface element include interface elements that do not meet the conditions, or it can be determined whether the N parent elements of the specified interface element include interface elements that do not meet the conditions, wherein the N parent elements can refer to the N parent elements traversed upward from the specified interface element in the ViewTree.

[0283] The electronic device 100 may analyze whether the parent element of the specified interface element includes an interface element that does not meet the conditions by comparing the page layout of the interface elements.

[0284] Among them, page layouts of interface elements that do not meet the conditions include: waterfall flow layout, scrolling page layout, Tab page layout, etc.

[0285] These layouts all belong to the layout of the application page, not the layout of the pop-up window. If the parent element of the specified interface element includes an interface element that does not meet the conditions, it means that the specified interface element is not an interface element in the pop-up window. Therefore, the electronic device 100 can end the search for the pop-up window; if the parent element of the specified interface element does not include an interface element that does not meet the conditions, it means that the specified interface element may be an interface element in the pop-up window. Therefore, the electronic device 100 can further search for other interface elements that may be in the pop-up window to determine whether the specified interface element is an element in the pop-up window.

[0286] Therefore, if the parent element of the designated interface element includes an interface element that does not meet the conditions, the electronic device 100 executes step S606; otherwise, the electronic device 100 executes step S603.

[0287] S603. The electronic device 100 searches for multiple interface elements that are associated with the specified interface element or the interface elements whose screen occupancy ratio is greater than a threshold value 2 found when traversing the interface elements, and determines these multiple interface elements as an element group.

[0288] These multiple interface elements can be regarded as the interface elements currently found that may be included in the pop-up window. In the embodiment of the present application, the element group composed of these multiple interface elements can also be called the first element group.

[0289] The electronic device 100 may search for the multiple interface elements in any of the following ways:

[0290] 1) Find these multiple interface elements according to the specified interface element

[0291] The multiple interface elements may include: a designated interface element, a sibling element of the designated interface element, a child element of the designated interface element, and a child element of the sibling element.

[0292] It is understandable that if the specified interface element does not contain child elements, then these multiple interface elements may not contain child elements of the specified interface element. Similarly, if the sibling element does not contain child elements, then these multiple interface elements may not contain child elements of the sibling element.

[0293] Taking the pop-up window shown in FIG7 as an example, if the designated interface element is View8, the multiple interface elements may include: View7, View8.

[0294] 2) Find these multiple interface elements based on the interface elements whose screen ratio is greater than the threshold 2 found when traversing the interface elements

[0295] Among them, these multiple interface elements may include all child elements of the parent element of the interface element whose screen ratio is greater than the threshold 2.

[0296] Taking the pop-up window shown in FIG7 as an example, if the interface element whose screen ratio is greater than the threshold 2 is View7, then the multiple interface elements may include: View7 and View8.

[0297] It is understandable that the electronic device 100 can also find these multiple interface elements in other ways, and the embodiment of the present application does not limit this.

[0298] S604. The electronic device 100 determines whether the element group meets the layout conditions of the pop-up window that needs to be adjusted.

[0299] Among them, the electronic device 100 can judge whether the element group meets the layout conditions of the pop-up window that needs to be adjusted based on the gap between the position, type, interaction event, width, height and other factors of each interface element in the element group and the pop-up window included in the pop-up window that needs to be adjusted.

[0300] Specifically, the electronic device 100 may use a scoring system similar to that mentioned in step S501 to determine a score for the element group, which indicates the likelihood that the element group is a pop-up window that needs to be adjusted. In other words, if the score meets certain requirements, the element group meets the layout conditions for the pop-up window that needs to be adjusted.

[0301] For example, Table 2 shows a scoring rule table for evaluating whether an element group meets the layout conditions of a pop-up window that needs to be adjusted, as provided in an embodiment of the present application.

[0302] Table 2

[0303] Among them, in the above Table 2:

[0304] (1) In Rule 1, if there are interface elements in the element group that exceed the screen display area, it means that there are some contents in the element group that are invisible to the user.

[0305] Furthermore, Rule 1 can be further limited to the specified interface element in the element group exceeding the screen display area. In this case, even if the element group is a pop-up window and some of its content exceeds the screen display area, the electronic device 100 will not identify it as a pop-up window that needs to be adjusted. Taking the specified interface element as a close control in a pop-up window as an example, if the text content in the pop-up window exceeds the display area, but the close control does not exceed the display area, although the user cannot view the complete content of the pop-up window, the pop-up window can still be quickly closed by the close control.

[0306] (2) In rule 2, a complex layout of an element group may mean that the number of element types contained in the element group exceeds a threshold of 6. For example, the threshold of 6 may be 10. This is because a pop-up window can only display limited content. Generally, a pop-up window contains fewer element types. Therefore, if the layout of the element group is complex, the possibility that the element group is a pop-up window can be reduced.

[0307] In addition, the specified type of interface elements may include radio buttons, check boxes, edit input boxes, etc. This is because pop-up windows generally do not contain radio buttons, check boxes, edit input boxes, etc. Therefore, if the element group contains these types of interface elements, the possibility of the element group being a pop-up window can be reduced.

[0308] (3) In rule 3, the number of interface elements with interactive events in the element group is too large, which may mean that the number of interface elements with interactive events in the element group is greater than a threshold of 7. For example, the threshold of 7 may be 4. This is because pop-ups generally only contain a few controls for interacting with users, such as close controls, cancel controls, and confirm controls. If there are too many interface elements with interactive events in the element group, the possibility that the element group is a pop-up window can be reduced.

[0309] (4) Rule 4 corresponds to Rule 3. The number of interface elements with interactive events is within a certain range, for example, the number does not exceed a threshold of 8. The threshold 8 can be equal to or less than the threshold 7. In other words, if there are fewer interface elements with interactive events in the element group, the possibility that the element group is a pop-up window can be increased.

[0310] (5) In rule 5, if the element group contains a large number of small-sized interface elements, the element group may not have a large-sized problem, and the pop-up window is generally displayed in the middle of the display screen. If some interface elements in the element group are not centered, the possibility that the element group is a pop-up window can be reduced. Among them, small size can mean that the width and / or height of the interface element is less than threshold 9, such as 350px, 400px. The embodiment of the present application does not limit the threshold 9. In addition,

[0311] (6) Rule 6 corresponds to Rule 5. If the element group contains a large number of interface elements with large sizes, the size of the element group may be large, which increases the possibility that the element group is a pop-up window that needs to be adjusted.

[0312] It can be understood that Table 2 is only an illustrative introduction to the example application of the scoring system rules in finding pop-up windows that need to be adjusted. In the embodiment of the present application, when determining whether an element group is a pop-up window that needs to be adjusted, the scoring system rules can also include more or fewer rules, and the scoring system rules can include other different rules, which is not limited in the embodiment of the present application.

[0313] Furthermore, after calculating the score of the element group using a scoring system similar to that in Table 2, the score can be used to indicate the likelihood that the element group is a pop-up window that requires adjustment. The higher the score, the higher the likelihood, and the lower the score, the lower the likelihood. If the score of the element group exceeds a threshold of 9, for example, 0, then the element group is determined to be a pop-up window that requires adjustment, i.e., step S605 is executed. Otherwise, the user interface does not include a pop-up window that requires adjustment, i.e., step S606 is executed.

[0314] S605. The electronic device 100 determines the element group as a pop-up window that needs to be adjusted.

[0315] The electronic device 100 determines that the element group is a pop-up window that needs to be adjusted, indicating that the element group belongs to the elements in the pop-up window that needs to be adjusted. After the electronic device 100 determines that the element group is a pop-up window that needs to be adjusted, the electronic device 100 can adjust the element group to achieve adjustment of the pop-up window.

[0316] Among them, the adjustment of the pop-up window may include: 1) shrinking the pop-up window; 2) adding a mapping interface element; 3) adjusting the position of the interface element. Specifically, shrinking the pop-up window may refer to shrinking each element in the element group and moving it to the center point of the element group; adding a mapping interface element may refer to adding a new interface element to the user interface and adding the interaction events of the interface elements in the element group that are outside the display area to the newly added interface element; adjusting the position of the interface element may refer to moving the interface elements in the element group that are outside the display area to the display area content. For specific information about the adjustment of the pop-up window, please refer to the above content and will not be repeated here.

[0317] In addition, for pop-ups where some interface elements are located outside the main content of the pop-up window, before shrinking the pop-up window, the pop-up window size is enlarged to adapt to the larger display area. Therefore, during the enlargement process of the pop-up window, the main content of the pop-up window may squeeze the display area of ​​these interface elements, causing the height or width of some interface elements to be 0. In this case, before shrinking the pop-up window, you need to adjust the height or width of these interface elements to make them greater than 0 before shrinking the pop-up window.

[0318] For example, taking this part of the interface elements including the close control 102B shown in Figure 1A and Figure 1B as an example, referring to the pop-up window 102 shown in Figure 1B, since the close control 102B is located below the pop-up window main body 102A, when the electronic device 100 draws the pop-up window 102 shown in Figure 1B, the pop-up window main body 102A will vertically squeeze the close control 102B, resulting in the height of the close control 102B being 0, so that the pop-up window 102 displayed by the electronic device 100 does not include the close control 102B.

[0319] The following describes the specific process of reducing a pop-up window by the electronic device 100 by taking the electronic device 100 adjusting the height of an interface element as an example.

[0320] Among them, Figure 13 is a schematic diagram of the process of reducing the pop-up window provided in an embodiment of the present application.

[0321] As shown in FIG13 , reducing the pop-up window may include the following steps:

[0322] S701. The electronic device 100 determines whether the height of the interface element outside the display area is 0.

[0323] In the embodiment of the present application, the interface element located outside the display area may refer to a first interface element.

[0324] If the height of the interface element is 0, the electronic device 100 executes step S702 ; otherwise, the electronic device 100 executes step S703 .

[0325] S702. The electronic device 100 adjusts the height of the interface element.

[0326] Specifically, the electronic device 100 can increase the height of the interface element. Accordingly, the electronic device 100 needs to reduce the height of other interface elements in the pop-up window that are arranged vertically with the interface element, so that the interface element and other interface elements in the pop-up window can be displayed simultaneously in the display area of ​​the pop-up window.

[0327] S703. The electronic device 100 reduces the size of the pop-up window.

[0328] In a specific implementation, the electronic device 100 can traverse each element in the element group, and scale each interface element in the element group based on the center point of the parent element of the top-level element in the element group, thereby reducing the pop-up window.

[0329] Exemplarily, taking the pop-up window 102 shown in FIG. 1A and FIG. 1B as an example, FIG. 14 shows a schematic diagram of a process in which the electronic device 100 adjusts the height of the interface element and reduces the size of the pop-up window.

[0330] As shown in (a) of FIG14 , the main portion 102A of the pop-up window occupies the display area of ​​the pop-up window, and the close control 102B exceeds the display area of ​​the pop-up window. At this time, the height of the close control 102B in the pop-up window display area is 0.

[0331] As shown in (b) of FIG. 14 , the electronic device 100 adjusts the relative heights of the pop-up window main portion 102A and the close control 102B so that the height of the close control 102B in the pop-up window display area is greater than 0.

[0332] As shown in (c) of FIG. 14 , the electronic device 100 further reduces the pop-up window shown in (b) of FIG. 14 , so that the electronic device 100 can display the complete pop-up window on the display screen.

[0333] It is understandable that part of the content of the pop-up window exceeds the display area, and the main content of the pop-up window may horizontally squeeze other interface elements, resulting in the width of some interface elements being 0. Therefore, the electronic device 100 can also determine whether the width of the interface element that exceeds the display area is 0. If the width of the interface element is 0, the electronic device 100 first adjusts the width of the interface element and then reduces the size of the pop-up window. The adjustment process is similar to steps S701-S703 and will not be repeated here.

[0334] In addition, in (b) of Figure 14, if the electronic device 100 further adjusts the relative heights of the pop-up window main body 102A and the close control 102B so that the close control 102B can be completely located within the pop-up window display area, the electronic device 100 can display the complete pop-up window without scaling the pop-up window.

[0335] That is to say, step S703 is an optional step. The electronic device 100 can adjust the relative height or relative width between the interface element that exceeds the display area and other interface elements in the pop-up window so that the interface element that exceeds the display area can be located in the display area, thereby omitting the step of the electronic device 100 scaling the pop-up window and reducing the amount of calculation of the electronic device 100.

[0336] S606. The electronic device 100 ends the search for the pop-up window.

[0337] It is understandable that after the electronic device 100 finishes searching for the pop-up window, the electronic device 100 may continue to monitor layout changes in the currently displayed user interface, and trigger a search for the pop-up window when the layout changes significantly.

[0338] In addition, it should be noted that in the embodiment of the present application, the pop-up window adjustment method can be a system function in the electronic device 100. In this way, the electronic device 100 does not need to pre-install third-party applications in advance, and can directly search and adjust pop-up windows in any application through the system unit. The application scenarios are more extensive, and it also facilitates user operations.

[0339] In general, the pop-up window adjustment method provided in the embodiment of the present application can find the pop-up window that needs to be adjusted in the user interface and adjust the pop-up window so that the adjusted pop-up window has a more appropriate size, or can make it more convenient for users to control the pop-up window, thereby ensuring the continuity of users browsing the page content, making the page content more beautiful, and improving the user experience.

[0340] FIG15 is a flow chart of a pop-up window adjustment method provided in an embodiment of the present application.

[0341] As shown in FIG15 , the pop-up window adjustment method may include:

[0342] S801. The electronic device 100 displays a first pop-up window, and a first interface element of the first pop-up window exceeds the display area of ​​the display screen.

[0343] Exemplarily, the first pop-up window may refer to the pop-up window 102 shown in Figure 1B, and the first interface element may refer to the close control 102B shown in Figure 1B, or the first pop-up window may refer to the pop-up window 202 shown in Figure 2B, and the first interface element may include the text information 202A and the allow control 202C shown in Figure 2B.

[0344] S802. The electronic device 100 adjusts the first pop-up window, and the adjustment includes: shrinking the first pop-up window, or displaying a second interface element in the display area, where the second interface element and the first interface element are used to trigger the same function, or moving the first interface element into the display area.

[0345] For example, the reduced first pop-up window may refer to the pop-up window 102 shown in Figure 1C, and the second interface element may refer to the close control 102C shown in Figure 1D, or the reduced first pop-up window may refer to the pop-up window 202 shown in Figure 2C, and the second interface element may refer to the allow control 202C shown in Figure 2D.

[0346] In some embodiments, when the electronic device 100 displays the first pop-up window, if the height of the first interface element is 0, the electronic device 100 may increase the height of the first interface element before reducing the first pop-up window.

[0347] Among them, increasing the height of the interface element before shrinking the pop-up window can ensure that the first interface element can be displayed in the user interface after the pop-up window is shrunk, ensuring that the electronic device 100 can adjust the content of the pop-up window outside the display screen to within the range of the display screen.

[0348] For a detailed description of how the electronic device 100 increases the height of the first interface element, please refer to the related description of the aforementioned steps S701-S703.

[0349] In addition, the first pop-up window can be displayed in the first user interface. Before the electronic device 100 adjusts the first pop-up window, the method also includes: the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, that is, searches for the first pop-up window in the first user interface.

[0350] That is to say, the electronic device 100 can evaluate the information of each interface element in the user interface to find the first pop-up window from the user interface, and then successfully adjust the pop-up window.

[0351] For a detailed description of the electronic device 100 searching for the first pop-up window, please refer to the aforementioned step S201 and its related contents.

[0352] Exemplarily, the first user interface may refer to the user interface 10 shown in FIG. 1B , or may refer to the user interface 20 shown in FIG. 2B .

[0353] Furthermore, in some embodiments, when the electronic device 100 displays the first pop-up window, the number of interface elements in the first user interface that undergo layout changes is greater than a second threshold, and the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window. Specifically, the electronic device 100 may include: when the number of interface elements in the first user interface that undergo layout changes is greater than the second threshold, determining the interface elements in the first user interface that belong to the first pop-up window. For example, the second threshold may refer to the threshold 1 mentioned in step S102 above.

[0354] In this way, the electronic device 100 can monitor the layout changes of the interface elements contained in the user interface in real time, and search for pop-up windows that need to be adjusted in the user interface based on the layout changes, so as to avoid the electronic device 100 being in a state of searching for pop-up windows all the time. The characteristics of pop-up windows as windows that are independent of the application page and pop up during the operation of the application are fully considered. When it is determined that a pop-up window may appear in the user interface, the search for the pop-up window is performed, thereby reducing the power consumption of the electronic device 100.

[0355] Furthermore, in some embodiments, the electronic device 100 may determine the interface elements in the first interface elements that belong to the first pop-up window within a preset time period.

[0356] In this way, the electronic device 100 can be prevented from spending too much time searching for pop-up windows, and the power consumption of the electronic device 100 can be controlled.

[0357] In some embodiments, the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the interface elements contained in the first user interface. Specifically, it may include: the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward, and the electronic device 100 determines the interface elements in the first user interface that belong to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

[0358] That is to say, when traversing interface elements, the electronic device 100 can traverse from the bottom layer of the ViewTree upwards. This is because if the user interface includes pop-up windows, in the ViewTree, the interface elements closer to the lower layer are drawn later and are more likely to be pop-up windows. Therefore, traversing from the bottom layer of the ViewTree upwards can minimize the number of interface elements traversed by the electronic device 100. When there are pop-up windows in the user interface, the interface elements that meet the requirements can be found as soon as possible, thereby improving the efficiency of the electronic device 100 in finding pop-up windows.

[0359] Furthermore, in some embodiments, after traversing the interface elements in the first user interface from the bottom layer of the ViewTree of the first user interface upward, the electronic device 100 may stop traversing after encountering an interface element whose size is greater than a third threshold. The third threshold may be Threshold 2 mentioned in step S406 above.

[0360] That is to say, when the electronic device 100 is traversing the interface elements, if it finds an interface element of a larger size, it can stop traversing. This is because if the user interface contains a pop-up window, when the electronic device 100 traverses to an interface element of a larger size, it means that most or all of the interface elements in the pop-up window may have been traversed. The electronic device 100 can end the traversal to avoid introducing interface elements that do not belong to the pop-up window into the calculation process during the search process, thereby saving search time.

[0361] Specifically, in the process of the electronic device 100 determining the interface element belonging to the first pop-up window in the first user interface, there may be the following two implementation methods:

[0362] 1) Search for pop-up windows through the whitelist

[0363] In this manner, the electronic device 100 may be pre-installed with a whitelist, which includes the IDs of the interface elements included in one or more pop-up windows.

[0364] Then, the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface based on one or more information of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface. Specifically, it may include: the electronic device 100 finds the third interface element in the first user interface that is in the whitelist; the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface through the third interface element.

[0365] In this way, the electronic device 100 can quickly determine whether the user interface contains a pop-up window that needs to be adjusted by searching whether the user interface contains an interface element in the whitelist.

[0366] For example, the third interface element can be the interface element corresponding to the background included in the first pop-up window. In this way, the interface element included in the first pop-up window can be a child element of the third interface element. In other words, the third interface element can be the highest-level element in the ViewTree of the first user interface among all the interface elements included in the first pop-up window.

[0367] For example, referring to FIG. 7 , taking the first pop-up window as the pop-up window 102 shown in FIG. 1B as an example, the third interface element may refer to the ViewGroup 6 shown in FIG. 7 .

[0368] 2) Find pop-up windows by specifying interface elements

[0369] In this way, the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface based on one or more information of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface. Specifically, it may include: the electronic device 100 searches for a specified interface element based on the multiple information of the interface elements contained in the first user interface; the specified interface element is the interface element contained in the pop-up window; thereafter, the electronic device 100 can determine the interface element belonging to the first pop-up window in the first user interface through the specified interface element.

[0370] For example, taking the pop-up window 102 shown in FIG. 1B as an example, the designated interface element may refer to a close control 102B.

[0371] That is, the electronic device 100 may first search for a landmark element in the pop-up window, and then use the landmark element to search for the pop-up window, so that the electronic device 100 can quickly find the pop-up window.

[0372] Further, in some embodiments, the electronic device 100 searches for a specified interface element based on the multiple information of the interface elements contained in the first user interface, which may specifically include: the electronic device 100 determines the scores of multiple interface elements in the first user interface based on the multiple information of the interface elements contained in the first user interface and the search for multiple information of the specified interface element, and the score of an interface element indicates the possibility that the interface element is the specified interface element; thereafter, the electronic device 100 may search for the specified interface element from the multiple interface elements based on the scores of the multiple interface elements.

[0373] Among them, the electronic device 100 calculates the score of the interface element through multiple information of the interface element, and can combine multiple factors to evaluate the similarity between the found interface element and the specified interface element from multiple aspects, thereby improving the accuracy and comprehensiveness of searching for the specified interface element.

[0374] For a detailed description of how the electronic device 100 searches for a specified interface element based on the score of the interface element, please refer to the relevant contents of the aforementioned steps S501-S504.

[0375] Exemplarily, the designated interface element can be any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before determining the scores of multiple interface elements in the first user interface, the electronic device 100 can also filter out multiple interface elements that contain interactive events from the interface elements in the first user interface.

[0376] In this way, when calculating the scores of interface elements, the electronic device 100 can only calculate the scores of the multiple filtered interface elements, and search for the specified interface element from these filtered interface elements, thereby reducing the amount of calculation of the electronic device 100 and speeding up the search speed of the electronic device 100.

[0377] Further, in some embodiments, the electronic device 100 determines the interface element belonging to the first pop-up window in the first user interface by specifying the interface element, which may specifically include: in the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, scrolling page layout, and Tab page layout, the electronic device 100 obtains a first element group, and the first element group includes: all child elements of the parent element of the interface element whose size is greater than the third threshold traversed by the electronic device 100 from the bottom layer of the ViewTree of the first user interface; when the first element group meets the layout conditions of the first pop-up window, the electronic device 100 determines the first element group as the interface element belonging to the first pop-up window in the first user interface.

[0378] Among them, evaluating the page layout of the parent element of the specified interface element can further determine whether the specified interface element searched by the electronic device 100 is the interface element in the pop-up window, thereby improving the accuracy of the pop-up window search.

[0379] Among them, whether the first element group contains the layout conditions of the first pop-up window can be determined by the electronic device 100 by calculating the score of the first element group based on the position, type, interaction event, width and height of the interface elements contained in the first pop-up window. The score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

[0380] It can be seen that by using multiple information of interface elements in the first pop-up window to calculate the score of the first element group, multiple factors can be used to evaluate the similarity between the first element group and the pop-up window from multiple aspects, thereby improving the accuracy and comprehensiveness of pop-up window search.

[0381] For a detailed description of how the electronic device 100 searches for a pop-up window by specifying an interface element, please refer to the relevant contents of the aforementioned steps S601-S606.

[0382] In some embodiments, the display screen of electronic device 100 may be a foldable screen. When electronic device 100 displays the first pop-up window, the foldable screen is in an unfolded state. In other words, this method can address the issue of pop-ups displayed on the foldable screen in the unfolded state exceeding the display area of ​​the display screen by adjusting the pop-up window. This makes the adjusted pop-up window easier for the user to read and operate, thereby improving the user experience.

[0383] FIG16 is a flow chart of another pop-up window adjustment method provided in an embodiment of the present application.

[0384] As shown in FIG16 , the pop-up window adjustment method may include:

[0385] S901. The electronic device 100 displays a first pop-up window, and the size of the first pop-up window is greater than a first threshold.

[0386] For example, the first pop-up window may refer to the pop-up window 102 shown in FIG1B , or the first pop-up window may refer to the pop-up window 202 shown in FIG2B . The first threshold may refer to the threshold 2 mentioned in step S406 . That is, if the first pop-up window includes an interface element with a large screen ratio, it indicates that the size of the first pop-up window is too large.

[0387] S902. The electronic device 100 adjusts the first pop-up window, where the adjustment includes: reducing the size of the first pop-up window.

[0388] Exemplarily, the first pop-up window after reduction may refer to the pop-up window 102 shown in FIG. 1C , or the first pop-up window after reduction may refer to the pop-up window 202 shown in FIG. 2C .

[0389] In some embodiments, the display screen of the electronic device 100 may be a foldable screen, and when the electronic device 100 displays the first pop-up window, the foldable screen is in an expanded state. In other words, this method can address the problem of overly large pop-ups displayed on the foldable screen when in the expanded state by reducing the size of the pop-up window, making it easier for the user to read the content within the pop-up window. It also prevents overly large pop-ups from causing controls within the pop-up window to be too dispersed, hindering user control of the pop-up window, and thus improving the user experience.

[0390] In addition, the first pop-up window can be displayed in the first user interface. Before adjusting the first pop-up window, the electronic device 100 can determine the interface elements in the first user interface that belong to the first pop-up window, and record the number of interface elements whose layout has changed in the first user interface, etc. For specific content not mentioned in steps S901-S902, please refer to the content in the aforementioned steps S801-S802.

[0391] FIG17 shows a schematic diagram of the hardware structure of the electronic device 100 .

[0392] The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, and / or a smart city device. The embodiments of the present application do not impose any particular restrictions on the specific type of the electronic device.

[0393] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0394] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0395] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0396] In some embodiments, the processor 110 may be configured to search for a pop-up window when the layout of the user interface changes significantly, and adjust the pop-up window after finding it. A detailed description of the search for a pop-up window and the adjustment of the pop-up window can be found in the aforementioned content and will not be repeated here.

[0397] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0398] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0399] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0400] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0401] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0402] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0403] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0404] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0405] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0406] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0407] In some embodiments, the display screen 194 may be used to display a pop-up window and a user interface in which the pop-up window resides. Detailed information about the user interface may be found in Figures 1A-1E and 2A-2E.

[0408] In addition, in the embodiment of the present application, if the electronic device 100 is a folding screen device, the electronic device 100 may include multiple display screens 194. Moreover, according to different opening and closing angles between the folding screens, the electronic device 100 may have different device forms.

[0409] For example, taking the electronic device 100 as a folding screen device, Figure 18 is a schematic diagram of the device form of the electronic device 100 provided in an embodiment of the present application. As shown in Figure 18, the electronic device 100 may include a display screen 194-1 and a display screen 194-2.

[0410] 18 (a) shows the electronic device 100 in a folded state. In the folded state, the angle between the display screen 194-1 and the display screen 194-2 is less than 180°, and the electronic device 100 can display images through the display screen 194-1. FIG18 (b) shows the electronic device 100 in an unfolded state. In the unfolded state, the angle between the display screen 194-1 and the display screen 194-2 is equal to 180°, and the electronic device 100 can display images through the combined display screen of the display screen 194-1 and the display screen 194-2.

[0411] 1A-1E and 18 , FIG1A is a user interface displayed by the electronic device 100 through the display screen 194 - 1 , and FIG1B-1E are user interfaces displayed by the electronic device 100 through the combined display screen of the display screen 194 - 1 and the display screen 194 - 2 .

[0412] 2A-2E and FIG. 18 , FIG. 2A is a user interface displayed by the electronic device 100 through the display screen 194 - 1 , and FIG. 2B-2E are user interfaces displayed by the electronic device 100 through the combined display screen of the display screen 194 - 1 and the display screen 194 - 2 .

[0413] It can be understood that the folding screen device provided in the embodiment of the present application may refer to an inward-folding folding screen device or an outward-folding folding screen device, wherein the inward-folding folding screen device folds the electronic device by folding inward, and the outward-folding folding screen device folds the electronic device by folding outward.

[0414] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0415] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0416] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0417] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0418] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0419] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0420] In some embodiments, the internal memory 121 can be used to store the number of interface elements in the user interface whose layout changes when the electronic device 100 monitors the layout changes of the user interface. In addition, the internal memory 121 can also be used to store a whitelist, interface elements that meet preset requirements found during the pop-up window search process, element groups found based on specified interface elements or interface elements with a size greater than a threshold 2, and so on.

[0421] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0422] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0423] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0424] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0425] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0426] In some embodiments, the electronic device 100 can detect the user operation on the pop-up window through the touch sensor 180K and perform corresponding operations. For example, the electronic device 100 can detect the user operation on the close control in the pop-up window through the touch sensor 180K and trigger the closing of the pop-up window.

[0427] The electronic device can be a portable terminal device equipped with iOS, Android, Microsoft or other operating systems, such as a mobile phone, tablet computer, wearable device, etc., and can also be a non-portable terminal device such as a laptop computer with a touch-sensitive surface or touch panel, a desktop computer with a touch-sensitive surface or touch panel. The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present invention takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100.

[0428] FIG19 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0429] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0430] The application layer can include a series of application packages.

[0431] As shown in FIG19 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0432] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0433] As shown in FIG19 , the application framework layer may include a pop-up window adjustment module, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0434] The pop-up window adjustment module may include: a layout monitoring module, a search module, and an adjustment module.

[0435] The layout monitoring module can be used to monitor layout changes of the user interface and record the number of interface elements in the user interface that have undergone layout changes.

[0436] The search module can be used to find pop-up windows in the user interface when the layout of the user interface changes significantly.

[0437] The adjustment module can be used to adjust the pop-up window after finding the pop-up window that needs to be adjusted in the user interface.

[0438] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0439] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0440] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0441] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0442] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0443] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0444] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0445] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0446] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0447] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0448] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0449] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0450] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0451] A 2D graphics engine is a drawing engine for 2D drawings.

[0452] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0453] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0454] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0455] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0456] The present application also provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program, and the processor may be used to call the computer program in the memory so that the electronic device executes the method executed by the electronic device 100 in any of the above embodiments.

[0457] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method performed by the electronic device 100 in any of the above embodiments.

[0458] In one possible design, the chip system further includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0459] The chip system can be composed of chips, or can include chips and other discrete devices.

[0460] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0461] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0462] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0463] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute any method executed by the electronic device 100 in any of the above embodiments.

[0464] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by any one of the electronic devices 100 in any of the above embodiments.

[0465] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-mentioned method.

[0466] In addition, embodiments of the present application further provide a device. Specifically, the device may be a component or module, and may include one or more processors and a memory connected to each other. The memory is configured to store a computer program. When the computer program is executed by one or more processors, the device performs the methods described in each of the above method embodiments.

[0467] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0468] The various implementation modes of this application can be combined arbitrarily to achieve different technical effects.

[0469] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0470] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0471] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pop-up window adjustment method, characterized in that: The method comprises: The electronic device displays a first pop-up window, and a first interface element of the first pop-up window exceeds a display area of ​​the display screen; The electronic device adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window, or displaying a second interface element in the display area, the second interface element and the first interface element are used to trigger the same function, or moving the first interface element to the display area.

2. The method according to claim 1, characterized in that When the electronic device displays the first pop-up window, the height of the first interface element is 0, and before shrinking the first pop-up window, the method further includes: Increase the height of the first interface element.

3. A pop-up window adjustment method, characterized in that: The method comprises: The electronic device displays a first pop-up window, where a size of the first pop-up window is greater than a first threshold; The electronic device adjusts the first pop-up window, and the adjustment includes: reducing the first pop-up window.

4. The method according to any one of claims 1 to 3, characterized in that: The first pop-up window is displayed in the first user interface, and before the electronic device adjusts the first pop-up window, the method further includes: The electronic device determines the interface element in the first user interface that belongs to the first pop-up window based on any one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface.

5. The method according to claim 4, characterized in that When the electronic device displays the first pop-up window, the number of interface elements in the first user interface whose layout changes is greater than a second threshold, Determining the interface element belonging to the first pop-up window in the first user interface specifically includes: When the number of interface elements in the first user interface that undergo layout changes is greater than a second threshold, the electronic device determines the interface elements in the first user interface that belong to the first pop-up window.

6. The method according to claim 4 or 5, characterized in that: The electronic device determines, within a preset time period, an interface element in the first user interface that belongs to the first pop-up window.

7. The method according to any one of claims 4 to 6, characterized in that: The electronic device determines, based on any one or more of the identification ID, attribute, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upwards; The electronic device determines the interface element belonging to the first pop-up window in the first user interface based on any one or more of the identification ID, attributes, width, height, and position of the traversed interface elements.

8. The method according to claim 7, characterized in that After the electronic device traverses the interface elements in the first user interface from the bottom layer of the view tree ViewTree of the first user interface upward, the method further includes: After traversing to an interface element whose size is greater than a third threshold, the electronic device stops traversing.

9. The method according to any one of claims 4 to 8, characterized in that: The electronic device is pre-installed with a whitelist, and the whitelist includes IDs of interface elements included in one or more pop-up windows; The electronic device determines, based on one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device finds a third interface element in the first user interface that is in the whitelist; The electronic device determines the interface element belonging to the first pop-up window in the first user interface through the third interface element.

10. The method according to claim 9, characterized in that The third interface element is an interface element corresponding to the background contained in the first pop-up window, and the interface element included in the first pop-up window is a sub-element of the third interface element.

11. The method according to claims 4-8, characterized in that: The electronic device determines, based on one or more of the identification ID, attributes, width, height, and position of the interface element contained in the first user interface, the interface element belonging to the first pop-up window in the first user interface, specifically including: The electronic device searches for a designated interface element based on the multiple pieces of information of the interface elements included in the first user interface; the designated interface element is an interface element included in the pop-up window; The electronic device determines the interface element belonging to the first pop-up window in the first user interface through the designated interface element.

12. The method according to claim 11, characterized in that The electronic device searches for a specified interface element based on the multiple pieces of information of the interface elements included in the first user interface, specifically including: The electronic device determines scores of a plurality of interface elements in the first user interface based on differences between the plurality of information of the interface elements included in the first user interface and the plurality of information of the designated interface element, wherein the score of one of the interface elements indicates a possibility that the interface element is the designated interface element; The electronic device searches for the designated interface element from the multiple interface elements based on the scores of the multiple interface elements.

13. The method according to claim 12, characterized in that The designated interface element is any one of a close control, a confirmation control, a cancel control, a reject control, and an allow control. Before the electronic device determines the scores of the plurality of interface elements in the first user interface, the method further includes: The electronic device filters out the multiple interface elements containing interactive events from the interface elements in the first user interface.

14. The method according to claim 13, characterized in that The electronic device determines, by using the designated interface element, an interface element in the first user interface that belongs to the first pop-up window, specifically including: In the ViewTree of the first user interface, when the page layout of the parent element of the specified interface element does not include any one of the waterfall layout, the scroll page layout, and the Tab page layout, the electronic device obtains a first element group, the first element group including: all child elements of the parent element of the interface element whose size is greater than a third threshold value that the electronic device traverses upward from the bottom layer of the view tree ViewTree of the first user interface; When the first element group meets the layout condition of the first pop-up window, the electronic device determines the first element group as interface elements in the first user interface belonging to the first pop-up window.

15. The method according to claim 14, characterized in that Whether the first element group meets the layout conditions of the first pop-up window is determined by the electronic device based on the score of the first element group calculated based on the position, type, interaction event, width, and height of the interface elements contained in the first pop-up window, wherein the score of the first element group indicates whether the first element group meets the layout conditions of the first pop-up window.

16. The method according to any one of claims 1 to 15, characterized in that: The display screen is a folding screen, and when the electronic device displays the first pop-up window, the folding screen is in an unfolded state.

17. An electronic device, characterized in that: The electronic device comprises a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method as claimed in any one of claims 1 to 16.

18. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 16.

Images